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Foreword 

Over  the  past  several  years,  it  has  become  increasingly  apparent  that  although  the  United 
States  Air  Force  (USAF)  buys  systems  in  isolation,  it  does  not  use  systems  in  isolation.  An  ever- 
changing  mix  of  systems,  which  enable  their  warfighting  capabilities,  supports  the  missions  of 
the  Air  Force.  In  an  ideal  world,  the  Air  Force  would  build  each  system  involved  to  satisfy 
specific  and  well-understood  requirements.  Then,  each  system  would  fit  into  its  pre-established 
USAF  role  supporting  whatever  capability  military  leaders  called  upon  for  action.  The  reality  is 
that  the  Air  Force  does  not  build  all  systems  through  a  homogenous  acquisition  and  development 
process,  it  does  not  use  all  systems  in  ways  foretold  at  their  inception,  and  not  all  systems  find 
themselves  used  among  predicted  interface  partners.  Especially  in  wartime,  the  exigencies  of 
war  sometimes  force  a  reconfiguration  among  systems  or  even  demand  systems  behave  in  ways 
that  create  new  capabilities.  When  such  changes  occur,  the  users  in  the  field  oftentimes  find  the 
tasks  associated  with  reengineering  interconnections  among  systems  falls  upon  them. 
Increasingly,  awareness  of  the  need  to  support  fungible  interconnection  among  systems  has 
driven  the  Air  Force  and  systems  engineers  to  start  thinking  about  the  demands  of  system-of- 
systems  configurations  and  the  engineering  issues  associated  with  building  and  supporting  them. 

The  System-of-Systems  Engineering  for  Air  Force  Capability  Development  study  was 
chartered  to  address  the  challenge  of  developing  systems-of-systems  that  are  more  effective.  The 
study  panel  conducted  this  study  in  response  to  a  request  by  the  Secretary  of  the  Air  Force  and 
the  Chief  of  Staff  of  the  Air  Force. 

In  response  to  their  direction,  the  System-of-Systems  Engineering  (SoSE)  study  team 
received  a  set  of  briefings  from  defense  industry  organizations,  Air  Force  operating  commands, 
other  Department  of  Defense  (DoD)  Agencies  and  organizations,  and  various  universities 
conducting  research  related  to  SoSE.  The  study  team  reviewed  numerous  briefings  and  other 
documents  from  Air  Force  and  Joint  organizations  concerning  system-of-systems  operations, 
acquisition,  and  development  procedures.  The  assistance  of  these  organizations  was  essential  to 
the  completion  of  our  effort.  Their  involvement  guided  the  study  team  toward  the  findings, 
concepts,  conclusions,  and  recommendations  that  comprise  this  study.  The  study  team  greatly 
appreciates  the  cooperation  of  these  organizations,  and  acknowledges  the  valuable  contributions 
their  efforts  made  to  this  study. 

The  undersigned  also  wish  to  acknowledge  the  outstanding  effort  put  forth  by  the  Air 
Force  Scientific  Advisory  Board  Secretariat,  the  members  of  the  System-of-Systems  Engineering 
study  team,  the  Study  Executive  Officers,  and  the  Technical  Writers  in  the  preparation  of  this 
study  -  whatever  value  is  found  in  this  work  is  attributable  to  them. 


Mr.  Thomas  “Skip”  Saunders 
System-of-Systems  Engineering  Study  Chairman 
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Executive  Summary 


Introduction 

Over  the  past  five  or  six  decades,  the  discipline  known  as  “Systems  Engineering”  has 
evolved.  At  one  time,  many  years  ago,  development  of  a  capability  was  relatively  simple  to 
orchestrate.  The  design  and  development  of  parts,  engineering  calculations,  assembly,  and 
testing  was  conducted  by  a  small  number  of  people.  Those  days  are  long  gone.  Teams  of 
people,  sometimes  numbering  in  the  thousands  are  involved  in  the  development  of  systems;  and, 
what  was  previously  only  a  development  practice  has  evolved  to  become  a  science  and 
engineering  discipline. 

Today,  engineers  and  developers  have  fairly  well  codified  the  processes  and  techniques 
for  building  large,  complex  systems,  and  when  executed  properly,  result  in  reliable  and  useful 
systems  that  serve  users  well.  The  challenges  of  today  involve  connecting  systems,  some  of 
which  are  rather  complex  systems,  together  into  system-of-systems  configurations.  This  activity 
has  not  yet  matured  to  the  point  of  declaring  it  a  discipline;  in  fact,  it  is  only  in  the  most 
rudimentary  phases  of  becoming  a  practice.  Consequently,  while  systems-of-systems  are  all 
around  us,  a  theory  of  engineering  applicable  to  systems-of-systems  has  yet  to  be  developed. 


The  Situation 

For  commercial  enterprise,  there  have  emerged  many  examples  of  systems-of-systems. 
Companies  can  oftentimes  serve  their  business  interests  when  they  can  form  various  formal  and 
informal  alliances.  Some  of  the  time,  these  alliances  are  manifest  as  protocols  that  can  be  used 
to  link  systems  to  other  systems.  One  example  is  the  emergence  of  common  usage  for  certain 
interconnection  standards.  For  this  form  of  standard,  we  have  chosen  a  catch  phrase 
“convergence  protocol”  to  characterize  the  fact  that  people  have  chosen  a  single  protocol  or 
standard  that  simplifies  connection  among  different  systems.  For  example,  Transmission 
Control  Protocol/Intemet  Protocol  (TCP/IP)  provides  a  commonly  accepted  convergence 
protocol  for  communication  among  applications  on  the  worldwide  web.  These  convergence 
protocols  are  observable  within  the  commercial  enterprise  world  in  many  forms  and  for  servicing 
many  different  customers.  Availability  of  these  convergence  protocols  has  made  it  much  easier 
for  organizations  to  generate  systems  that  they  can  connect  to  other  systems  and  has  thereby 
made  it  easier  to  access  a  much  broader  market  than  would  be  available  if  they  only  used  private 
or  individual  custom  protocols. 

Some  of  the  existing  protocols  are  electronic,  some  are  information  system  centric,  and 
some  are  physical.  Consider  the  common  electrical  outlet  plug,  or  the  widespread  use  of  S-video 
connections  among  entertainment  products.  Yet  another  example  is  the  emergence  of  common 
packaging  standards  that  facilitate  freight  transport  in  containers  that  the  airline,  rail,  sea,  and 
trucking  transportation  systems  can  easily  move.  There  are  commercial  forces  derived  from 
business  association,  profit  motive,  and  market  access,  which  provoke  the  evolution  of 
convergence  protocols. 

Adherence  to  standard  protocols  allows  systems  such  as  televisions  to  interface  to  other 
devices  such  as  digital  video  cameras,  computers,  and  so  forth  such  that  a  capability,  which 
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transcends  what  any  one  could  do  on  its  own,  becomes  available  to  the  users  of  the  three 
component  systems.  It  is  noteworthy  (and  an  important  distinction  for  what  constitutes  system- 
of-systems  versus  merely  a  complex  system)  to  consider  that  a  television  is  a  system  in  its  own 
right.  It  serves  in  isolation  a  useful  function  to  many,  and  it  can  be  developed  as  a  stand-alone 
product.  Similarly,  both  video  cameras  and  computers  serve  useful  functions  for  their  owners, 
and  can  operate  in  isolation  from  the  other  components  of  a  system-of-systems. 

For  the  U.S.  Air  Force,  the  challenges  of  building  a  system-of-systems  are  particularly 
important  because  many  of  the  systems  already  developed  can  function  as  contributors  to  the 
performance  of  other  systems.  However,  smooth  and  simple  assembly  of  a  system-of-systems  is 
quite  difficult  for  the  DoD.  We  can  trace  much  of  this  difficulty  to  the  absence  of  militarily 
useful  convergence  protocols.  Where  civil/commercially  generated  convergence  protocols  are 
available,  the  DoD  can  derive  benefit.  Nevertheless,  even  in  these  cases,  the  DoD  does  not 
always  consider  the  ramifications  of  ignoring  widely  used  convergence  protocols  versus 
adopting  unique  connectivity  among  systems.  The  DoD  has  habitually  emphasized  the  pre¬ 
determination  of  interconnections  among  systems,  and  has  therefore  designed  and  implemented 
connectivity  via  custom  interconnections. 

However,  the  times  are  changing.  The  USAF  (as  well  as  other  parts  of  the  DoD)  is  now 
very  sensitive  to  the  need  for  spontaneous  interconnection  among  systems  previously  thought 
unrelated.  For  example,  in  Desert  Storm,  information  from  space  sensors  was  determined  to  be 
useful  in  queuing  air  and  missile  defense  assets.  However,  lack  of  common  protocols  among 
those  systems  forced  a  long  engineering  effort  to  make  the  relevant  systems  interoperate. 

Increased  emphasis  on  “capabilities”  and  “effects”  has  likewise  migrated  thinking  from  a 
platform  centric  perspective  to  one  that  embraces  the  mutual  benefits  of  having  systems  work 
together  in  synergy  to  achieve  a  desired  objective  capability  or  even  a  surprising  beneficial 
emergent  behavior. 


Findings 

As  the  study  began,  it  quickly  became  apparent  that  there  was  confusion  among  many 
people  over  what  constituted  a  “system-of-systems.”  Consequently,  study  panel  devoted  some 
effort  towards  establishing  a  common  vernacular  and  a  framework  for  discussing  the  issues 
surrounding  engineering  a  system-of-systems.  Secondly,  it  also  was  quickly  apparent  that  the 
forces  that  create  system-of-systems  alliances  in  the  commercial  sector  are  not  always  applicable 
within  the  DoD.  The  profit  motive  does  not  apply;  moreover,  there  are  specific  contravening 
directives  against  any  program  manager  who  would  consider  incorporating  features  that  might 
contain  speculative  benefits.  (In  contrast,  speculatively  including  capabilities  in  a  commercial 
system  is  an  often  times  well  rewarded  market  share  enhancement  strategy.) 

Further,  investigation  of  the  state  of  maturity  for  engineering  a  system-of-systems 
revealed  that  there  was  little  in  terms  of  codified  practice  or  discipline  that  could  be  adapted  for 
use  within  the  DoD.  Although  there  are  many  examples  in  the  commercial  world,  and  a  few 
examples  within  DoD,  a  common  development  approach  is  not  apparent. 
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Discussion 

Definitions  establish  a  common  vernacular  and  allow  discussion  to  proceed  with  common 
understanding  among  the  participants.  For  the  purposes  of  this  study: 

A  System-of-Systems  (SoS)  is  defined  as:  A  configuration  of  systems  in  which 
component  systems  can  be  added/removed  during  use;  each  provides  useful  services  in 
its  own  right;  and  each  is  managed  for  those  services.  Yet,  together  they  exhibit  a 
synergistic,  transcendent  capability. 

Systems  Engineering  (SE)  is  defined  as:  The  process  by  which  a  customer’s  needs  are 
satisfied  through  the  conceptualization,  design,  modeling,  testing,  implementation,  and 
operation  of  a  working  system. 

System-of-Systems  Engineering  (SoSE)  is  defined  as:  The  process  of planning, 
analyzing,  organizing,  and  integrating  the  capabilities  of  a  mix  of  existing  and  new 
systems  into  a  system-of-systems  capability  that  is  greater  than  the  sum  of  the 
capabilities  of  the  constituent  parts.  This  process  emphasizes  the  process  of 
discovering,  developing,  and  implementing  standards  that  promote  interoperability 
among  systems  developed  via  different  sponsorship,  management,  and  primary 
acquisition  processes. 

It  is  assumed  that  each  component  system  has  been  well-engineered  using  traditional 
Systems  Engineering  methodologies  as  a  precursor  to  the  formation  of  a  system-of-systems.  One 
important  aspect  of  “goodness”  for  the  application  of  traditional  Systems  Engineering  is  that  the 
perspectives  of  seven  critical  stakeholders  (i.e.,  user,  acquirer,  developer,  tester,  trainer, 
sustainer,  and  researcher)  are  considered  throughout  that  process.  These  seven,  essentially  peer, 
stakeholders  are  not  always  included  in  processes  that  purport  to  modify,  improve,  or  otherwise 
streamline  the  acquisition/development  processes  of  the  DoD.  This  can  be  a  serious  omission.  It 
often  results  in  shortened  or  ineffective  life  cycles  for  products  that  take  shortcuts.  As  will 
become  apparent,  we  propose  provision  for  incorporating  these  stakeholder  perspectives  as  part 
of  the  methodology  for  System-of-Systems  Engineering.  This  will  be  accomplished  by 
embedding  the  stakeholder  perspectives  in  the  fielding  steps  for  systems  that  participate  in  the 
proposed  methodology. 

The  new  methodology  emphasizes  four  considerations:  the  human  role,  discovery  and 
application  of  convergence  protocols,  motivation  issues,  and  experimentation  venues. 


The  Role  of  the  Human  in  a  System-of-Systems 

Whenever  the  Air  Force  generates  a  system-of-systems,  interaction  among  the  systems 
often  includes  human-to-human  interactions.  If  the  machine-to-machine  aspect  of  SoS  is  weak, 
then  it  falls  upon  the  humans  to  achieve  the  interaction.  This  can,  and  often  does,  create  a  very 
challenging  environment  for  the  human;  sometimes  leading  to  missed  opportunities  or  serious 
mistakes.  The  lack  of  sound  Human  System  Interface  designs  can  exacerbate  this.  Coordinated 
situation  awareness  is  difficult  to  manage  if  the  individual  systems  miss  or  convey  confusing  or 
conflicting  information  to  their  operators. 
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Assuming  there  are  sound  human  systems  interfaces  for  individual  systems,  the  Air  Force 
can  greatly  reduce  the  burden  on  the  human-to-human  coordination  if  effective  inter-system 
interoperation  is  established.  It  is  the  objective  of  our  study  to  suggest  ways  for  improving  inter¬ 
system  interoperation  at  the  hardware  and  software  level  while  keeping  in  mind  that  sometimes 
the  human-to-human  interaction  is  the  appropriate  interface.  An  effective  system-of-systems 
will  promote  collaborative  decision-making  and  shared  situation  awareness  amongst  the  human 
operators.  This  occurs  by  addressing  the  need  for  common,  consistent  human-system  interfaces. 


Discovery  Olid  Application  of  Convergence  Protocols 

A  highly  successful  convergence  protocol  for  DoD  application  in  a  system-of-systems 
configuration  is  the  use  of  discovery  technologies  for  information  management  among  its 
members.  One  of  the  key  factors  dictating  the  achievement  of  system-of-systems  configurations 
that  support  network  centric  operations  is  the  availability  of  mechanisms  that  promote 
information  sharing  among  systems.  Direct  system-to-system  linkages  presuppose  that  the 
systems  know  a  priori  about  each  system  that  might  benefit  from  any  other  system.  Many 
combat  situations  have  disproved  this  assumption.  Because  of  this,  a  key  technology  that  needs 
focused  attention  is  intelligent  agents  to  discover  and  better  manage  information  synergies  in 
dynamic  systems-of-systems.  While  there  is  substantial  development  of  intelligent  agents  in 
commercial  markets,  bom  of  the  competition  in  internet  search  engines,  this  effort  will  not  meet 
the  needs  of  DoD  systems-of-systems  because  of  militarily  unique  requirements  for  security, 
commander's  intent,  and  resource  prioritization.  As  a  result,  the  pace  of  development  of  this 
technology  for  military  systems  continues  to  be  a  limitation  for  realizing  effective  systems-of- 
systems. 


Motivation  Mechanisms 


The  motivations  within  the  commercial  market  do  not  easily  translate  into  the  DoD 
environment.  Whereas  those  in  the  commercial  market  reward  innovation  and  risk  taking  with 
financial  and  market  access  benefits,  those  within  the  DoD  environment  hold  risk  aversion  and 
predictability  in  much  higher  esteem.  The  traditional  approach  for  enforcement  within  the  DoD 
is  promulgation  of  directives,  guidance,  instructions,  and  so  forth.  Oddly  enough,  there  is  no 
shortage  of  such  documents  encouraging  good  interoperability  among  systems.  However,  there 
is  insufficient  specificity  and  detail  within  that  guidance  for  individual  program  managers  to 
know  exactly  what  the  DoD  is  asking  them  to  do.  The  guidance  expresses  the  vision  of  inter¬ 
operation  and  coordinated  systems-of-systems  very  well  and  repeatedly.  However,  the  tangible 
steps  a  program  manager  must  accomplish  to  embed  his/her  product  with  readiness  to  participate 
in  SoS  configurations  is  absent. 

There  is  a  further  danger.  Premature  enforcement  of  standards  can  do  more  harm  than 
good.  For  example,  use  of  the  Ada  programming  language,  and  specific  directives  to  use 
products  from  the  Defense  Integration  Infrastructure  -  Common  Operating  Environment  (DII- 
COE)  were  declared  prior  to  when  they  could  be  reliably  and  practically  adopted.  Finding  that 
“effective  middle  ground”  via  documented  guidance,  instruction,  and  directive  is  nearly 
impossible  (in  our  study  team’s  estimation).  Consequently,  we  are  suggesting  an  alternative  in 
which  the  DoD  acquisition  process,  program  managers,  and  other  stakeholders  can  encourage  the 


PUBLIC  RELEASE 
4 


PUBLIC  RELEASE 


adoption  of  System-of-Systems  Engineering  practices  as  a  part  of  the  normal  “user 
requirements”  management  process. 


Experimentation 

The  use  of  experimentation  with  three  very  specific  and  differing  objectives,  we  believe, 
can  activate  the  “other”  most  powerful  enforcement  approach  available  to  the  DoD:  “user 
requirements.”  Aside  from  directives  mentioned  above,  DoD  has  deeply  engrained  the 
satisfaction  of  user  requirements  within  its  bureaucratic  construct  for  acquiring  and  developing 
systems.  Therefore,  we  propose  a  three-pronged  approach  that,  if  properly  administered,  will 
dovetail  with  existing  laws,  regulations,  and  procedures  for  building  systems  that  will  be  SoS 
enabled. 

The  first  experimentation  venue  is  devoted  to  disrupting  conventional  wisdom  concepts 
of  operation  (CONOPS).  While  individual  systems  are  fabricated  with  the  objectives  of 
addressing  existing  or  newly  considered  CONOPS,  the  need  for  synergistic  behavior  all  too  often 
“surprises”  the  user  when  they  are  engaged  in  wartime  activities.  Repeatedly,  only  within  the 
“laboratory”  of  wartime,  do  users  apply  innovative  thinking  forced  by  activities  of  an  enemy. 
That  innovative  thinking  then  forces  users  to  consider  connecting  assets  in  ways  that  they  never 
tried  before,  never  trained  for  before,  and  perhaps  never  tested  or  designed  before.  The 
consequence  is  rapid  ad  hoc  lashing  together  of  systems  in  a  situation  of  extremis  that  may  or 
may  not  prove  beneficial  to  the  lives  of  combatants.  Our  suggestion  is  to  use  an  experimentation 
venue  (not  unlike  that  which  was  originally  proposed  for  the  Joint  Expeditionary  Force 
Experiment  or  JEFX  experimentation  process)  to  explore  innovative  CONOPS.  It  is  important  to 
note  that  JEFX  is  not  providing  this  service  today.  It  has  drifted  towards  a  training  and 
demonstration  focus  that  is  unable  to  tolerate  the  intentionally  disruptive  experimentation  that  we 
suggest  as  essential  to  success.  Oddly  enough,  while  the  peacetime  DoD  emphasizes  training 
and  predictable  behavior  of  its  systems,  in  wartime  DoD  needs  innovation  as  its  most  important 
leadership  and  execution  attribute.  Training  of  innovation  skills  is  precisely  what  DoD  and  the 
Air  Force  should  explore  with  the  kind  of  experimentation  we  suggest;  yet  there  is  no  current 
venue  for  accomplishing  that  training.  There  are  existing  assets  (such  as  the  Distributed  Mission 
Operations  Center,  DMOC)  that  possess  simulation  facilities  and  computer  modeling  capabilities 
that  could  allow  much  experimentation  along  the  lines  we  suggest  without  incurring  the  costs 
associated  with  JEFX;  yet  these  assets  are  not  currently  being  applied  in  that  manner.  Modeling 
and  data  collection  during  experimentation  can  enable  analyses  over  widely  varying  scenarios 
and  configurations  that  will  reveal  the  value  of  inter-operability,  not  just  at  the  network  level,  but 
also  at  the  system- to- system  application  level.  Therefore,  “disruptive  and  innovative  CONOPS 
experimentation”  is  the  first  experimentation  venue  that  we  believe  the  Air  Force  should  activate. 

The  second  experimentation  venue  is  a  technical  one.  The  Air  Force  needs  to  debate  the 
relative  merits  of  candidate  convergence  protocols  in  an  atmosphere  of  tangible  experience. 
Preferences  for  one  or  another  approach  can  be  debated  in  open  forum,  but  there  is  no  substitute 
for  hands-on  experience  and  evaluation.  Only  hands-on  evaluation  can  recognize  when  a 
protocol  is  sufficiently  mature  for  incorporation  within  systems  that  must  have  reliable  behavior 
and  predictable  benefits.  Likewise,  we  must  incorporate  precaution  against  premature 
dissemination  of  standards.  Consequently,  the  Air  Force  should  devote  the  second 
experimentation  venue  to  the  distillation  of  specific  protocol  standards  that  program  managers 
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can  incorporate  within  systems  under  development.  These  specific  protocols  should  be  defined 
and  characterized  in  sufficient  detail  to  achieve  convergence  protocol  behavior,  but  should  not  be 
defined  as  specific  vendor  products.  It  is  not  appropriate  to  specify  specific  vendor  products,  but 
rather  much  more  effective  to  specify  specific  protocol  algorithms.  The  Joint  Photographic 
Experts  Group  (JPEG)  standard  for  pictorial  imagery  is  an  excellent  example  of  the  kind  of 
specificity  imagined.  The  decoding  algorithm  is  the  expression  of  this  standard.  An  encoded 
artifact  is  JPEG  compliant  if  the  well-codified  JPEG  decode  algorithm  is  able  to  create  a  picture 
out  of  that  artifact.  The  standard  is  mute  with  respect  to  the  encoding  algorithm.  Thus,  there  is 
plenty  of  leeway  for  competitors  to  derive  efficient,  fast,  and  high  compression,  or  specially 
tailored  algorithms  and  to  compete  among  one  another  for  market  superiority.  Yet  the  standard 
has  universal  application  since  regardless  of  the  (even  proprietary  in  some  instances)  JPEG 
compression  applied,  the  decode  algorithm  works  successfully.  Therefore,  we  believe 
“convergence-protocol-evaluation”  experimentation  is  the  second  experimentation  venue  the  Air 
Force  should  activate. 

The  third  experimentation  venue  is  one  focused  on  rapid  fielding  of  SoS  enabled  systems. 
More  than  one  legitimate  path  may  be  followed  for  the  generation  of  systems  that  serve  the  needs 
of  the  IJSAF.  Some  of  these  are  specifically  established  as  means  for  streamlined  systems 
development,  and  some  are  specifically  established  to  allow  user  communities  to  maintain 
flexibility  and  responsiveness  to  emerging  immature  needs.  These  alternative  acquisition 
approaches  commonly  do  not  take  into  account  the  full  life  cycle  of  a  product;  consequently, 
systems  brought  to  the  field  with  these  alternative  approaches  seldom  have  long  lives.  Were 
there  an  effective  and  supportive  set  of  advisors  available  to  these  development  teams,  the 
leveraging  implied  by  incorporation  of  the  seven  critical  stakeholders  mentioned  earlier,  could 
make  these  “alternative  source”  products  far  more  effective.  From  the  SoS  perspective,  it  would 
allow  some  of  the  “convergence  protocols”  derived  in  the  second  experimentation  venue  to  be 
incorporated  from  the  outset  in  these  streamlined  acquisition  products.  Synergy  among 
disciplined  practitioners  of  the  seven  stakeholders  with  the  speed  to  field  objectives  of  the  system 
developers  could  prove  beneficial  to  all  parties.  There  are  many  details  to  be  worked  out  here, 
but  the  affected  participants  have  people  who  are  creative  and  innovative  within  their 
organizations.  We  have  confidence  that  under  the  proper  leadership  an  effective  solution 
providing  mutual  benefit  would  enable  SoS  options  for  the  USAF. 


State  of  SoS_  Engineerings  Practice 

Lastly,  the  study  panel  held  numerous  discussions  with  others  about  the  maturity  of 
SoSE.  There  are  schools  beginning  to  establish  curricula  devoted  to  the  science  of  SoSE.  These 
schools  are  investigating  mechanisms  that  underlie  SoS  opportunities.  For  example,  what  are  the 
metrics  that  the  Air  Force  can  apply  to  systems  for  evaluating  the  benefits  of  incorporating 
provisions  for  reconfiguration,  flexibility,  inter-operability,  and  so  forth  that  affect  SoSE?  Of 
relevance  to  these  programs,  the  Air  Force  needs  a  body  of  investigation  and  research  for 
discovering  and  developing  convergence  protocols.  At  a  more  general  level,  a  theoretic 
foundation  for  a  predictive  analytic  SoSE  discipline  is  needed. 
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Recommendations 

We  have  four  recommendations  that  fall  into  two  categories: 

The  first  category  is  that  of  technology  exploration.  Here,  we  should  1)  establish  the 
experimentation  venues  as  a  means  for  achieving  pragmatic  near-term  results.  As  described 
above,  these  venues  should  support  disruptive,  innovative  CONOPS  experiments,  convergence 
protocol  evaluation  and  selection,  and  support  for  rapid  provisional  fielding  of  SoS  enabled 
systems.  Also  from  a  pragmatic  approach,  technology  that  provides  discovery  among  systems 
and  makes  association  of  information  needs  among  the  publishers  and  subscribers  is  necessary  to 
achieve  the  full  potential  of  network  centric  operations.  Therefore,  we  should  2)  pursue  these 
enabling  technologies. 

The  second  category  is  that  of  infrastructure  enhancement.  Here,  we  recommend  that  3) 
the  USAF  task  the  Air  Force  research  establishment  to  promote  investigation  within  this  topic. 
There  are  aspects  of  such  investigation  that  would  be  applicable  to  both  commercial  and  DoD 
environments,  but  there  are  some  aspects  of  such  investigation  that  must  be  focused  on  DoD  (and 
USAF)  unique  constraints.  The  differences  in  enterprise  environments  between  commercial 
interests  and  DoD  legal  constraints  can  have  a  profound  effect  upon  the  direction  of  research. 
Only  DoD-sponsored  research  is  likely  to  produce  results  that  are  usable  within  the  DoD.  To 
facilitate  this  properly,  we  need  the  involvement  of  the  seven  critical  stakeholders.  Therefore, 
we  also  recommend:  4)  creation  of  an  infrastructure  that  would  allow  full  two-way  participation 
of  the  seven  critical  stakeholders.  The  goal  here  is  not  only  to  facilitate  their  participation  in  SoS 
development,  but  also  to  provide  the  opportunity  for  them  to  incorporate  the  new  SoSE 
methodology  into  their  respective  bureaucracies. 


Summary 

The  SAB  study  into  System-of- Systems  Engineering  has  developed  a  set  of 
recommendations  devoted  to  a  pragmatic  approach  to  the  issues  of  building  component  systems 
capable  of  assembly  as  a  system-of-systems.  We  will  not  fix  the  problem  of  “lack  of  a  defined, 
developed,  and  applied”  SoS  Engineering  discipline  via  one  report.  However,  we  believe 
adoption  of  the  recommendations:  institution  of  the  experimentation  venue,  adoption  of 
management  strategies  that  translate  SoS  convergence  protocols  into  user  requirements,  and 
sponsorship  of  additional  research  and  development  (R&D)  will  help  the  Air  Force  move  in  a 
healthy  direction  as  well  as  act  as  a  first  step  towards  building  a  SoSE  discipline.  The  theoretic 
perspective  for  SoSE  has  not  yet  been  established,  but  there  are  several  capable  R&D 
organizations  standing  ready  within  the  academic  community  and  elsewhere.  These 
organizations  should  be  encouraged  to  pursue  further  study  into  this  topic. 
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Study  Charter 


■  Propose  a  Systems  Engineering  Methodology 

■  Tailored  for  AF  use 

■  Suitable  for  Software  Intensive  Systems  that  include 
humans  as  part  of  system 

■  For  Robust  and  adaptable  SoS  that: 

■  Provides  validated  “useful”  operational  capabilities 

■  Can  be  delivered  at  a  cycle  time  matching  operational  tempo  needs 

■  Leverage 

■  Existing  SE  principles 

■  Existing  “non-traditional”  practices 

■  Operator  expectations  of  SE 

■  Evolving  research  in  SoS  engineering 

■  Find  an  exemplar  program 


This  study  was  chartered  to  examine  the  state  of  Systems  Engineering  practice  for 
developing  a  system-of-systems.  We  were  asked  to  propose  a  methodology  that  the  USAF  could 
apply  when  assembling  software  intensive  systems  into  configurations  that  offered  useful 
operational  capabilities,  and  to  do  so  at  a  pace  that  would  be  responsive  to  operational  needs. 
We  were  asked  to  take  advantage  of  lessons  from  any  quarter  to  include  both  traditional  and 
innovative  practices.  Lastly,  we  were  asked  to  see  if  we  could  find  an  exemplar  program  that 
might  benefit  from  our  proposed  new  methodology. 
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Problem  Statement 


■  Most  systems  are  acquired  individually 

■  Yet,  capabilities  are  achieved  with  assemblies  of  multiple  systems 

■  The  unanticipated  need  for  system  to  system  interactions  too  often  require 
clever,  ad  hoc  work-arounds 


■  Although  weak  “systems  engineering”  is  sometimes  blamed  for 
disconnects  experienced  in  the  field. ..the  real  problem: 

A  SoS  Engineering  discipline  has  never  been  defined ,  developed ,  or 

applied 

\j  xi 


Battlefield  Consequences: 

■  Can’t  take  full  advantage  of  assets 

■  Capabilities  are  “Late  to  Need” 

■  Unanticipated  CONOPS  “work-arounds”  developed  on  the  fly 

■  User  has  to  compensate  for  weak  interoperability  designs 


The  Air  Force  acquires  most  of  its  systems  individually,  to  serve  the  needs  of  a  specific 
user  for  a  specific  purpose.  Yet,  mission  capabilities  are  usually  the  result  of  contributions  from 
multiple  users  using  multiple  systems.  Sometimes,  new  capabilities  sought  would  be  possible  if 
existing  systems  were  to  work  together.  However,  since  the  Air  Force  may  not  have  known  a 
priori  that  it  would  use  the  systems  in  the  service  of  the  novel  capability,  these  systems  require 
creative  work-arounds  to  be  able  to  exchange  information  and  work  together. 

It  is  not  “weak  Systems  Engineering”  at  fault;  rather,  it  is  the  absence  of  System-of- 
Systems  Engineering.  None  exists. 

Experiences  on  the  battlefield  have  revealed  this  problem  for  many  years.  For  example, 
in  Desert  Storm,  information  from  space  sensors  was  determined  to  be  useful  in  queuing  air  and 
missile  defense  assets.  However,  lack  of  common  protocols  among  those  systems  forced  a  long 
engineering  effort  to  make  the  relevant  systems  interoperate.  More  recently,  the  concept  of 
operations  for  using  the  Global  Hawk  shifted  dramatically  once  its  value  in  theater  was 
recognized.  The  user,  all  too  often,  becomes  the  de  facto  systems  engineer,  acquirer,  developer, 
tester,  and  sustainer  for  capabilities  such  as  imagery  in  the  cockpit. 
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So,  Why  System-of-Systems  * 


Now? 


■  There  is  a  clear  trend: 

■  Away  from  a  platform  centric  view  of  systems 

■  Towards  capabilities  and  effects 

■  Capabilities  and  effects  are  based  on  collections  of 
systems  working  synergistically 

■  Some  systems  support  multiple  capabilities 

■  We  need  systems  built  in  ways  that  facilitate 
cooperation...  in  ways  that  make  them  capable  of 
operating  as  a 


Svstem-of-Svstems 


Why  has  this  topic  surfaced  now?  We  can  trace  part  of  the  interest  to  the  notion  of 
Transformation,  especially  transformation’s  emphasis  on  capabilities.  Capabilities  are  usually 
dependent  upon  a  mix  of  systems. 

However,  there  is  no  “one-for-one  mapping”  among  systems  and  the  capabilities  they 
provide.  In  other  words,  just  as  some  capabilities  depend  on  multiple  systems,  some  systems 
support  multiple  capabilities.  The  solution  of  placing  one  person  “in  charge”  of  a  capability  runs 
the  risk  that  some  systems  would  fall  under  the  auspices  of  multiple  “in  charge”  capability 
managers.  So,  there  is  an  expectation  (or  at  least  a  hope)  that  some  form  of  Systems  Engineering 
practice  encompassing  both  system  development  and  system-of-systems  development  levels  will 
facilitate  the  synergistic  assembly  of  systems  into  something  that  achieves  useful,  operationally 
relevant  capabilities.  We  conducted  this  study  to  examine  these  possibilities. 
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Bottom  Line  Up  Front  (BLUF) 


m  A  “System-of-Systems”  is  different  from  a  “System” 

■  SoS  Engineering 

■  involves  managing  future  uncertainty 

■  not  a  well  codified  discipline  (yet),  it  is  a  practice 

■  Commercial  world  motivations  do  not  translate  easily  to  DoD  world 

■  depends  on: 

■  Well  engineered  systems  PLUS 

■  New  processes  to  define  and  leverage  user  requirements  such  that 
systems  are  SoS  enabled 

■  Recommendations 

■  Establish  three  experiment  venues 

■  Build  technologies  that  support  convergence  protocols  interoperation 

■  Promote  research  into  discipline  and  theory  of  SoSE 

■  Institute  management  approaches  that  motivate  7  critical  systems 
engineering  stakeholders  to  embrace  SoS  solutions 


As  has  become  a  tradition  for  SAB  studies,  we  disclose  the  punch  line  up  front.  In  the 
case  of  this  study,  the  punch  line  is  “experimentation.”  Although  SoS  engineering  has  not 
matured  enough  to  be  a  true  discipline,  practices  have  appeared  that  seem  to  nurture  system-of- 
systems  development.  Those  practices  are  a  combination  of  well-practiced  traditional  Systems 
Engineering,  plus  the  emergence  of  certain  technical  products  and  architectural  styles  prevalent 
in  information  systems.  What  we  are  seeking  is  some  means  of  accelerating  what  seems  to 
happen  naturally,  but  at  too  slow  a  pace.  We  have  become  convinced  that  the  Air  Force  could 
influence  (accelerate)  the  pace  of  system  maturity  and  evolution  of  important  systems  attributes 
via  judicious  use  of  experimentation. 

We  also  capitalize  on  some  of  the  recommendations  from  our  “sister  study”  “Domain 
Integration”  chaired  by  Dr  Pete  Worch  and  Professor  Alex  Levis.  Most  notably,  our  study 
ratifies  their  suggestion  that  certain  service  oriented  architecture  constructs  are  beneficial  for 
achieving  domain  interoperation.  Such  constructs  are  also  important  for  the  rapid  assembly  of  a 
system-of-systems. 

We  also  point  out  that  System-of-Systems  Engineering  depends  upon  motivating  and 
engaging  the  critical  stakeholders  in  traditional  Systems  Engineering  processes.  We  do  not 
discard  traditional  Systems  Engineering  in  favor  of  a  new  methodology;  rather,  it  needs  to  be 
augmented  to  support  development  of  a  system-of-systems. 

Therefore,  our  recommendations  are  the  establishment  of  three  new  experimentation 
venues  for  the  USAF  to  achieve  critical  insight  into  the  benefits  and  challenges  of  a  system-of- 
systems.  It  is  difficult  to  motivate  engagement  in  a  concept  unless  the  benefits  are  readily 
apparent  to  the  proposed  beneficiaries.  That  is  a  long-winded  way  of  saying  “You  can  lead  a 
horse  to  water,  but  can’t  make  it  drink.”  Users  will  not  embrace  system-of-systems  solutions  if 
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they  do  not  perform  a  useful  capability.  Moreover,  and  perhaps  most  important,  the  benefits  of 
complying  with  a  system-of-systems  architecture  will  not  be  apparent  if  the  flexibility  achieved 
with  that  architecture  are  not  understood.  The  foundational  support  for  including  system-of- 
systems  enabling  requirements  within  individual  systems  will  not  happen  unless  such  flexibility 
is  valued.  We  therefore  suggest  that  an  experimentation  venue  that  forces  users  to  consider 
innovative  CONOPS  will  be  an  important  part  of  developing  healthy  systems-of-systems.  The 
JEFX  experimentation  venue  might  appropriately  be  changed  to  perform  as  this  venue. 
Operationally  oriented  experiments  will  help  users  learn  how  to  respond  to  threat  environments 
with  flexible  application  of  existing  systems.  Next,  the  technical  and  systems  engineering 
community  needs  a  second  experimentation  venue  to  define  and  gain  practical  technical 
experience  with  what  we  might  call  “market  enabling  convergence  protocols.”  This  is  the 
“technical  Aha!”  part  of  the  USAF  SAB  investigation,  and  mostly  described  by  the  “Domain 
Integration”  study  report.  Lastly,  we  propose  an  experimentation  venue  for  the  acquisition 
community  that  they  can  use  in  a  practical  sense  to  accelerate  the  movement  of  new  systems 
capabilities  to  the  field.  We  will  introduce  the  notion  of  “provisional  fielding”  as  a  means  for 
sponsoring  both  improved  traditional  Systems  Engineering  discipline  for  the  systems  we 
currently  develop  as  stovepipe  development  efforts  as  well  as  for  including  some  of  the  loose 
couplers  described  in  the  “Domain  Integration”  study  among  all  systems  which  may  become 
candidate  components  within  a  system-of-systems  configuration.  We  recommend  Air  Combat 
command  and  the  Deputy  Chief  of  Staff  of  the  Air  Force  for  Plans  and  Operations  (AF/XO) 
pursue  the  CONOPS  experimentation  venue.  The  Air  Force  Research  Lab,  the  Assistant 
Secretary  of  the  Air  Force  for  Acquisition  (SAF/AQ),  and  Secretary  of  the  Air  Force,  Office  of 
Warfighting  Integration  and  Chief  Information  Officer  (SAF/XC)  should  work  together  to  pursue 
infrastructure  experimentation.  SAF/AQ  should  also  pursue  experimentation  for  provisional 
fielding. 

The  second  recommendation  is  for  the  development  of  some  specific  technical 
capabilities  that  allow  systems  to  discover  the  presence  of  other  systems  relevant  to  their 
mission.  Too  often,  information  is  available  in  “the  enterprise”  that  does  not  reach  all  the 
potential  beneficiaries.  For  example,  agent  technology  that  provides  performance  advocated  by 
the  2000  SAB  study  on  the  Joint  Battlespace  Infosphere  (JBI)  (Ref:  SAB-TR-99-02)  needs 
accelerated  development.  There  is  also  a  need  for  investment  in  modeling  tools  to  support  the 
kind  of  disruptive-innovative  concept  of  operation  experimentation  we  advocate.  In  particular, 
understanding  network  behavior  in  an  environment  of  system-of-systems  operation  would  be 
useful  to  investigate.  We  recommend  responsibility  for  this  go  to  the  Commander,  Air  Force 
Research  Laboratory  (AFRL/CC). 

The  third  recommendation  is  for  the  development  of  a  much  more  disciplined  approach 
to  understanding  the  engineering  issues  surrounding  systems-of-systems.  This  study 
concentrated  on  pragmatic  steps  the  USAF  might  follow  for  improving  its  experiences,  but  there 
is  a  need  for  a  more  theoretic  investigation  that  would  result  in  the  ability  to  conduct  predictive 
analysis  and  behavior  estimates  for  an  emerging  SoSE  discipline.  We  believe  responsibility  for 
this  should  go  to  the  Air  Force  Office  of  Scientific  Research  (AFOSR). 

The  last  recommendation  is  for  the  pursuit  of  management  approaches  that  engage  the 
critical  seven  stakeholders  in  any  Systems  Engineering  methodology.  Motivational  forces  are 
needed  within  each  to  participate  in  successful  pursuit  of  SoSE.  So,  building  alliances  among 
the  different  organizations  and  cultures  affected  by  SoSE  would  likely  contribute  strongly  to 
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improved  outcomes  for  system-of-systems.  We  recommend  that  SAF/XC  should  take 
responsibility  for  forming  the  IPT.  Further,  we  recommend  that  IPT  membership  should  include 
the  following: 

Air  Combat  Command  (ACC) 

Air  Force  Space  Command  (AFSPC) 

Assistant  Secretary  of  the  Air  Force  for  Acquisition  (SAF/AQ) 

Air  Force  Operational  Test  and  Evaluation  Center  (AFOTEC) 

Air  Education  and  Training  Command  (AETC) 

Air  Force  Materiel  Command  (AFMC) 

Air  Force  Research  Laboratory  (AFRL) 
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Panel  Members  and  Staff 


SAB  Members  and  Consultants 

Mr  Thomas  F  "Skip"  Saunders 

Dr  Wanda  Austin 

Dr  John  Brock 

Mrs  Natalie  Crawford 

Dr  Mica  Endsley 

Mr  Ed  Glasgow 

Dr  Daniel  Hastings 

Dr  Alexander  Levis 

Dr  Richard  Murray 

Dr  Donna  Rhodes 

Dr  Marvin  Sambur 

Ms  Heidi  Shyu 

Mr  Philip  Soucy 

Dr  David  Whelan 

General  Officer  Participant - 

Lt  Gen  (S)  Charles  E  Croom  Jr. 

SAB  Staff  - 

Maj  Alan  Seraile 
Maj  Christopher  Berg 
Maj  David  Herring 
Mr  Justin  Waters 


The  MITRE  Corporation 

The  Aerospace  Corporation 

Northrop  Grumman  Corporation 

The  RAND  Corporation 

SA  Technologies 

Lockheed  Martin  Corporation 

Massachusetts  Institute  of  Technology 

George  Mason  University 

California  Institute  of  Technology 

Massachusetts  Institute  of  Technology  (Consultant) 

Raptors  Consulting  Group 

Raytheon  Corporation 

Modem  Technology  Solutions,  Inc 

The  Boeing  Company 


Director,  C4ISR  Infostructure,  SAF/XCI 


Executive  Officer 
Program  Manager 
Technical  Writer 
Technical  Assistant 


The  study  panel  was  comprised  of  a  set  of  people  with  diverse  backgrounds  and 
experience,  but  each  of  who  brought  perspectives  important  to  our  investigation  of  this  rather 
complex  subject.  Having  people  with  such  outstanding,  Government,  military,  industrial,  and 
academia  backgrounds  and  credentials  helped  maintain  focus  on  what  we  hope  are  useful  and 
innovative  recommendations  for  the  USAF. 
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The  Aerospace  Corporation 


Headquarters  Air  Force 

SAF/AQRE 


FFRDCs 


Universities 

California  Institute  of  Technology 
George  Mason  University 
Massachusetts  Institute  of  Technology 
Old  Dominion  University 
Purdue  University 
Stevens  Institute  of  Technology 
University  of  California  San  Diego 
University  of  Southern  California 
University  of  Virginia 


MITRE 


We  went  to  many  places  and  discovered  that  a  large  number  of  people  and  organizations 
are  giving  considerable  thought  to  the  issues  and  benefits  of  assembling  a  system-of-systems. 
Moreover,  there  is  substantial  resource  investment  taking  place  that  offers  potential  to  leverage 
support  for  some  of  the  more  “resource  intensive”  recommendations  we  make.  In  general,  we 
were  encouraged  during  our  information  gathering  visits  for  the  prospects  of  System-of-Systems 
Engineering. 
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1.  Definitions 


Definitions 


■  Experiment  (ik-'sper-e-ment): 

■  Something  to  do  when  you  don’t  know  the  outcome 

■  Demonstration  (de-men-'strA-shen): 

■  Something  to  do  when  you  want  to  show  a  skeptic  what  you 
already  believe 

■  Requirements  (r-'kwlr-mantz): 

■  What  you  say  when  you  know  what  you  want 

■  Systems  ('si s-temz): 

■  What  systems  engineers  build  to  satisfy  requirements 

■  Surprise  synergy  (se(r)-'prlz  si-ner-jE): 

■  What  you  get  when  you  knew  you  wanted  something  good,  but 
didn’t  know  how  to  ask  for  it 

■  Systems  of  Systems  ('sis-temz  av  ’sis-temz): 

■  What  SoS  engineers  build  to  achieve  surprise  synergy 


A  few  definitions  are  worth  considering.  First,  the  distinction  between  experiment  and 
demonstration  is  critically  important.  Misunderstanding  of  the  purpose  of  an  experiment  would 
severely  disrupt  the  ability  to  execute  the  recommendations  of  this  study. 

Similarly,  misunderstanding  the  ultimate  contribution  from  sound  System-of- Systems 
Engineering  would  easily  allow  the  Air  Force  and  DoD  to  misinterpret  our  recommendations  as 
“activities  already  underway”.  We  wish  to  emphasize  that  predicted  behavior  should  be  the 
product  of  sound  Systems  Engineering.  Tolerance  for  future  needs  is  the  objective  of  sound 
System-of- Systems  Engineering.  Achieving  “surprise  synergy”  is  the  objective  whereby  users 
discover  systems  in  the  field  are  interoperable  even  though  developers  did  not  design  those 
specific  systems  that  way  at  the  time  of  a  system’s  development. 
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What  is  a  System-of-Systems?  orof 
Distinguishing  Reai  From  “Faux” 

■  From  small  stovepipes  to  large  stovepipes  -  is  NOT 
systems  of  systems 


■  Component  systems  can  be  added/removed  during 
use;  each  provides  useful  services  in  its  own  right; 
and  each  is  managed  for  those  services. 

■  Yet,  together  they  exhibit  a 
synergistic  capability. 

Ref  Dr  Alexander  Levis 


Our  discussion  of  System-of-Systems  Engineering  (SoSE)  depends  upon  clear 
differentiation  of  the  System-of-Systems  (SoS)  from  the  large-scale  system.  A  common 
misconception  is  that  a  transition  from  small  stovepipes  to  very  large  stovepipes  implies  an  SoS; 
this  is  incorrect.  A  true  SoS  is  comprised  of  component  systems  that  users  can  add  or  remove 
during  use,  with  each  providing  useful  services  in  its  own  right,  and  they  can  manage  each  for 
those  services.  The  SoS  is  created  when  these  are  combined  in  such  a  way  as  to  deliver  a 
capability,  enabled  by  the  synergy  of  the  component  systems.  Such  capability  cannot  be 
delivered  by  the  simple  summation  of  the  standalone  capabilities  of  the  SoS’s  component 
systems. 

It  is  beneficial  to  consider  Dr  Mark  Maier’s  characterization  of  distinguishing  features  for 
an  SoS.  These  attributes  become  important  because  of  the  criteria  for  applying  some  of  the 
engineering  practices  we  propose  are  most  appropriate  for  SoS,  and  are  probably  not  so  relevant 
to  systems  that  are  merely  “large  and  complex.”  (For  these  large  and  complex  systems, 
traditional  Systems  Engineering  should  be  sufficient,  assuming  it  is  properly  applied.) 

Maier  (1996)1  highlights  five  characteristics  that  distinguish  the  SoS  from  very  large 
complex  monolithic  systems: 

1.  Operational  Independence  of  the  Elements:  If  the  SoS  is  disassembled  into  its 
component  systems,  they  are  still  able  to  operate  independently  in  a  useful  manner. 

2.  Managerial  Independence  of  the  Elements:  Component  systems  are  separately 
acquired  and  continue  to  be  managed  independently. 


1  Maier,  M.,  “Architecting  Principles  for  Systems-of-Systems”,  Proceeding  of  the  6th  Annual  INCOSE  Symposium,  p.  567-574, 
1996. 
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3.  Evolutionary  Development:  The  SoS  evolves  over  time,  with  component  systems 
capabilities  added,  removed,  or  modified  as  needs  change  and  experience  is  gained. 

4.  Emergent  Behavior:  The  SoS  has  emergent  capabilities  and  properties  that  do  not 
reside  in  the  component  systems. 

5.  Geographic  Distribution:  The  SoS  component  systems  are  geographically  distributed 
but  have  the  ability  to  readily  exchange  information. 


PUBLIC  RELEASE 

23 


PUBLIC  RELEASE 


System-of-Systems  Engineering ; 

Two  Perspectives 


■  A  “capability”  perspective: 

■  System-of-systems  engineering  is  the  interdisciplinary, 
CROSS-SYSTEMS  process  that  ensures  the  development 
and  evolution  of  CAPABILITIES  to  meet  multiple 
stakeholders’  evolving  needs  across  periods  of  time  that 
exceed  the  lifetimes  of  its  individual  systems 


Ref:  Dr.  Jeremy  Kaplan;  1st  Annual  conference  on  SoSE 


■  A  “connecting  the  parts”  perspective: 

■  System-of-systems  engineering  is  the  process  of 
discovering,  developing,  and  implementing  standards 
that  promote  interoperability  among  systems  developed 
via  different  sponsorship,  management,  and  primary 
acquisition  processes 


The  set  of  systems  comprising  a  system-of-systems  includes  independently  capable 
systems,  that  when  integrated,  deliver  significantly  greater  capability.  A  single  system  (or  less 
than  full  combination  of  all  involved  systems)  cannot  provide  the  capability  achieved  by  the 
system-of-systems.  There  are  two  legitimate  perspectives  relevant  to  the  engineering  of  the  SoS: 
(1)  the  capability  oriented  perspective  and  (2)  the  “connecting  the  parts”  perspective. 

Capability  Oriented  Perspective:  This  first  perspective  relates  to  the  overall  process 
for  establishing  new  capabilities  via  an  SoSE  process  with  continuing  evolution  over  the  SoS  life 
cycle  as  the  needs  and  opportunities  for  delivering  new  capabilities  arise.  Thus,  the  SoSE 
process  must  provide  for  mechanisms  to  discover  and  promote  new  capabilities  and  synergies 
that  arise  in  yet  unrealized  systems-of-systems  while  establishing  the  impact  of  new  CONOPS 
on  the  performance,  utility,  and  development  of  the  constituent  systems.  This  perspective  also 
recognizes  that  the  SoSE  process  must  be  in  harmony  with  the  traditional  SE  life  cycle 
development  processes  of  the  individual  component  systems  endeavors. 

“Connecting  the  Parts”  Oriented  Perspective:  The  second  perspective  recognizes  that 
the  SoS  is  built  from  a  collection  of  independently  acquired  and  operating  systems  that  must  now 
be  connected  together.  Therefore,  there  must  be  specific  mechanisms  that  allow  the  component 
systems  to  come  together.  Interoperability  standards  are  the  primary  mechanism  to  enable 
effective  connection  of  the  parts,  enabled  by  an  environment  that  fosters  standardization.  It  is 
important  to  point  out  that  consistent  protocol  behavior  is  the  property  that  enables 
interoperability.  Adherence  to  a  specific  product  is  often  over-constraining  and  provokes 
problems  with  respect  to  version  upgrades.  A  carefully  crafted  definition  of  the  protocol  is  a 
better  means  for  a  long-lived  approach  towards  establishing  interoperability  among  systems  that 
may  be  developed  during  different  timeframes,  from  different  sources,  and  from  differing 
organizations  or  missions.  As  a  result,  an  important  element  of  future  SoSE  implementation  will 
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be  the  identification,  development,  and  verification  of  robust  interoperability  standards  that 
through  testing  and  time  become  unassailable  "must  haves"  in  each  individually  procured 
system. 
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■  One  big  system  with  lots  of  subsystems 

■  This  is  just  traditional  Systems  Engineering 


-75% 

Think  this  way 


•  We  know  how  to  do  it 

■  We  don’t  always  do  it  the  way  we  know  how. . .  (that’s  another  problem) 

■  Lots  of  cooperating  systems  — ==ZZ7  -20% 

*  We  know  in  advance  they  should  play  well  together  Think  this  way 

■  We’ll  just  build  them  in  a  way  that  allows  them  to  play  together 

■  network  enabled  is  a  good  first  step...  they  all  play  on  the  network 

■  Systems  that  will  be  brought  together  in  the  field 

■  It  is  a  “pick-up”  game,  it  will  always  be  a  pick-up  game,  needs  will  change 

■  Wish  they  played  together...  but  who  could  have  predicted  they  would  need 
to  interact? 

■  ...surprise  synergy.. .can  we  build  to  support  ultimate  network  centricitv? 


Researchers  can  find  multiple  definitions  of  System-of-Systems  Engineering  in  the 
literature.  This  study  has  shown  that  there  are  at  least  three  unique  views  on  SoSE,  and  that  few 
people  are  thinking  about  SoSE  in  a  manner  that  will  yield  the  desired  outcomes  for  Air  Force 
systems. 

It  is  concerning  that  approximately  75%  of  subject  matter  experts  consulted  in  the  study 
viewed  an  SoS  as  just  a  big  system  with  lots  of  subsystems,  with  a  perspective  that  it  requires 
only  traditional  SE.  This  perspective  is  “we  know  how  to  do  it... we  don’t  always  do  it  the  way 
we  know  how“  (that  is  another  problem). 

About  20%  of  the  subject  matter  experts  consulted  in  the  study  offered  a  more 
progressive  view.  These  people  viewed  an  SoS  as  many  cooperating  systems,  where  we  know  in 
advance  that  they  should  play  well  together.  The  approach  is  to  build  them  in  a  way  that  allows 
them  to  play  together  with  network  enabling  as  the  good  first  step. 

Only  about  5%  had  the  desirable  perspective  of  SoS  as  collaborative  systems  that  will  be 
brought  together  in  the  field,  recognizing  it  as  a  “pick-up”  game  that  will  always  be  a  pick-up 
game  as  needs  will  change.  In  this  view,  the  perspective  is  that  the  SoS  involves  many  legacy 
systems  that  we  “wish  they  played  together,  but  who  could  have  predicted  they  would  need  to 
interact?”  In  this  view,  there  is  “surprise  synergy”  and  the  challenge  is  perhaps  to  build  to 
support  ultimate  network  centricity. 

A  shift  in  mindset  will  be  necessary  to  move  the  industry  forward  to  thinking  about  SoSE 
as  a  discipline  that  builds  upon  traditional  Systems  Engineering  approaches,  but  involves  new 
approaches  and  enablers  to  evolve  and  sustain  a  system-of-systems. 
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Pre  Condition:  Systems  Engineering  is 

Effectively  Applied  to  the  Individual  sv  W  /? 
Systems  Within  the  SoS 


Systems  Engineering  is  the  process  by  which  a  customer’s  needs 
are  satisfied  through  the  conceptualization,  design,  modeling, 
testing,  implementation,  and  operation  of  a  working  system 


jages  an  :ie  contributors  to  a  system’s  life-cycle 
Users  -  people  in  the  field  who  benefit  from  system 
Developers  -  people  who  construct  the  system 
Acquirers  -  pt  ople  who  contract  for  and  pay  for  system 
Testers  -  peoj:  le  who  evaluate  system  for  suitability 
Trainers  -  people  who  ensure  users  know  how  to  use  it 
Sustainers  -  people  who  keep  system  up  to  date 
Researcher..  -  people  who  provide  next  generation  ideas 


Considers  external  tutors  -  threat,  risksCadversaries^pr  market 
competitors  which  influence  the  sy^tsnfsaTJIIily  lu 'Support 
customer  needs 

- Seven  (+1)  Critical  Stakeholders 


Systems  Engineering  is  the  process  by  which  a  customer’s  needs  are  satisfied  through  the 
conceptualization,  design,  modeling,  testing,  implementation,  and  operation  of  a  working  system. 
It  is  a  full  life  cycle  discipline.  It  engages  all  contributors  in  the  system’s  life  cycle  including 
users,  developers,  acquirers,  testers,  trainers,  sustainers,  and  researchers.  In  any  Systems 
Engineering  methodology,  it  is  important  to  consider  the  perspectives  of  each  of  these 
stakeholder  communities.  Too  often,  what  appear  to  be  good,  new  methodologies  (e.g.,  Pre- 
Planned  Product  Improvement,  Evolutionary  Acquisition,  Spiral  Acquisition,  and  so  forth)  run 
into  difficulties  because  balance  among  these  seven  stakeholders  is  not  included.  This  is  not 
necessarily  the  fault  of  the  new  methodology  concept;  it  is  more  often  the  fault  of  the 
implementation.  If  the  SoSE  methodology  proposed  here  is  to  be  successful,  its  implementation 
needs  to  have  participation  from  each  of  these  stakeholders. 

In  addition,  good  Systems  Engineering  also  considers  external  factors  including  threats, 
risks,  adversaries,  or  market  competitors  influencing  the  system’s  ability  to  support  stakeholder 
needs. 


Many  standards  and  guidebooks  today  describe  the  Systems  Engineering  process  and 
practices.  In  addition,  recent  studies  of  Systems  Engineering  in  the  defense  industry  have 
concluded  that  its  practices  are  sound,  although  not  always  effectively  applied.  SE 
Revitalization  policies2  issued  by  the  DoD  and  Air  Force  target  the  unresolved  issues  of 
insufficient  attention  to  and  application  of  Systems  Engineering  in  programs. 

The  Systems  Engineering  process  is  applied  to  the  development  of  complex  systems,  as 
well  as  to  development  of  the  subsystems  that  comprise  it.  A  full  life  cycle  process  invokes  the 


2 


SE  Revitalization  policies  and  recent  guidance  can  be  found  on  the  Air  Force  Center  for  Systems  Engineering  website: 
http://cse.afit.edu 
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contributions  of  the  multi-disciplinary  team  at  appropriate  points  in  the  life  cycle  and  includes 
milestone  reviews.  In  an  SoS,  this  traditional  Systems  Engineering  process  continues  to  be 
applied  to  the  component  systems  comprising  the  SoS  but  it  is  not  sufficient  in  itself  for  SoSE. 
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What  Differentiates  Engineering  of  a 
System  Vice  a  System-of-Systems?  4JO 

/(  4dv‘vs0<S 


Traditional  Systems  Engineering 

System-of-Systems  Engineering 

Purpose 

Development  of  single  system  to  meet 
stakeholder  requirements  and  defined 
performance 

Evolving  new  system-of-systems  capability 
by  leveraging  synergies  of  legacy  systems 

System 

Architecture 

System  architecture  established  early  in 
lifecycle  and  remains  relatively  stable 

Dynamic  reconfiguration  of  architecture  as 
needs  change;  use  of  service  oriented 
architecture  approach  as  enabler 

System 

Interoperability 

Defines  and  implements  specific  interface 
requirements  to  integrate  components  in 
system 

Component  systems  can  operate 
independently  of  SoS  in  a  useful  manner 
Protocols  and  standards  essential  to 
enable  interoperable  systems 

System 

“ilities” 

Reliability,  Maintainability,  Availability  are 
typical  ilities 

Added  “ilities”  such  as  Flexibility, 

Adaptability,  Composeability 

Acquisition  and 
Management 

Centralized  acquisition  and  management 
of  the  system 

Component  systems  separately  acquired 
and  continue  to  be  managed  as 
independent  systems 

Anticipation  of 
Needs 

Concept  phase  activity  to  determine 
system  needs 

Intense  concept  phase  analysis  followed  by 
continuous  anticipation,  aided  by  ongoing 
experimentation 

In  1961,  Simon  Ramo,  addressed  the  First  Systems  Symposium  at  Case  Institute  of 
Technology.  In  that  address,  he  remarked  that  the  outcome  of  the  race  between  Systems 
Engineering  versus  the  rapidly  increasing  complexity  of  our  growing  civilization  would  be 
“determined,  in  effect,  by  whether  Systems  Engineering  as  a  discipline  is  able  to  grow  and 
develop  quickly  enough  to  successfully  meet  the  problems  of  our  future.”3  While  Systems 
Engineering  has  continued  to  evolve  and  mature,  we  face  the  same  challenge  for  a  discipline  that 
can  address  complex  SoS  problems.  A  successful  outcome  will  be  the  development  of  an  SoSE 
discipline  that  will  complement  but  not  replace  the  traditional  SE  practices.  Traditional  SE 
remains  essential  to  development  of  the  individual  product  systems  and  subsystems,  the  building 
blocks  from  which  a  system-of-systems  is  created.  The  table  above  compares  some  of  the  facets 
of  traditional  SE  to  SoSE. 

In  contrast  to  the  engineering  of  a  single  system,  the  discipline  of  SoSE  must 
accommodate  larger  scope  and  greater  complexity  of  integration  efforts,  collaborative 
engineering  processes  in  an  extended  enterprise,  and  engineering  under  conditions  of  high 
uncertainty.  Decision-making  may  involve  difficult  decisions  such  as  a  delay  of  individual 
legacy  system  upgrades  to  respond  to  urgent  needs  to  deliver  capability  or  improved 
performance  in  the  overall  SoS.  Architecture  becomes  critically  important  with  the  SoS 
requiring  the  ability  to  reconfigure  the  architecture  dynamically  to  respond  to  changing  needs 
and  threats,  as  well  as  to  the  availability  of  new  system  technologies.  Understanding  and 
responding  to  emergent  SoS  behavior  -  both  positive  and  negative  -  is  another  important  aspect 
of  SoSE.  To  address  such  challenges,  traditional  Systems  Engineering  is  necessary  but  not 
sufficient  for  the  engineering  of  a  system-of-systems. 


3  Ramo,  S.,  forward  to  Systems:  Research  and  Design,  Proceedings  of  the  First  Systems  Symposium  at  Case  Institute  of 
Technology,  ed.  Donald  P.,  Eckman  (New  York,  1961) 
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Comparing  traditional  SE  and  SoSE  from  the  perspectives  of  seven  key  stakeholder 
groups  enables  further  understanding  of  changes  in  approach  and  methods  for  SoSE,  as  shown  in 
the  table  below. 


Table  1.  Comparison  of  Stakeholder  Perspectives  for  Traditional  SE  versus  SoSE 

Traditional  Systems  Engineering 

System-of-Systems  Engineering 

User 

Defined  set  of  people  in  the  field  who 
benefit  from  system  with  relatively  well 
understood  expectations. 

Changing  set  of  people  in  the  field  who 
benefit  from  evolving  system-of-systems  in 
response  to  dynamic  needs. 

Developer 

System  developed  by  prime  contractor  and 
subcontractors  with  single  program  office. 
High  degree  of  focus  on  the  integration  of 
the  system  components  to  specified  level 
of  performance. 

Development  is  distributed  with  collaboration 
of  multiple  contractors  and  suppliers, 
responding  to  multiple  program  offices. 
Component  systems  may  be  in  varied  life 
cycle  phases.  Effective 
human  systems  integration  is  critical. 

Trainer 

Trainers  with  high  degree  of  knowledge  of 
system  deliver  training  based  on  system 
intended  use;  training  changes 
incrementally  when  system  upgrades  are 
delivered. 

Trainers  need  to  understand  highly  complex 
and  evolving  SoS;  training  will  need  to  be 
personalized  based  on  dynamic  needs  of  sets 
of  stakeholders  at  needed  point  in  time. 

Tester 

Verification  and  validation  in  accordance 
with  V-model  of  system  to  well  specified 
set  of  requirements  and  documented 
needs;  master  test  plan  drives  testing 
activities. 

Will  involve  increased  use  of  models  and 
simulations  and  use  of  experiments  to 
formulate  validation  approach.  Requires  high 
degree  of 

coordination  of  cross  system  testing  activities. 

Sustainer 

Sustainment  needs  and  requirements 
planned  in  front  end  of  systems  effort; 
sustainment  activities  performed  by  single 
organization. 

Sustainment  needs  and  requirements  will 
evolve  as  SoS  evolves;  high  degree  of 
coordination  across  multiple  sustainment 
organizations  is  required. 

Acquirer 

Sound  acquisition  practices  used  to  select 
and  manage  prime  system  contractor. 

Direct  relationship  between  acquisition 
and  contractor  program  offices. 

Complex  planning  and  negotiations  will  be 
necessary  for  SoS,  as  well  as  coordinated 
acquisition  of  component  system  capabilities. 
Some  centralization  of  SoSE  management  via 
use  of  a  Lead  Systems  Integrator  role. 

Researcher 

SE  research  efforts  are  typically  program 
focused;  performed  in  program  laboratory 
involving  modeling  and  simulation;  and 
research  is  performed  in  early  phases  of 
system  development  or  directed  at  specific 
system  enhancements  in  later  phases. 
Research  is  directed  at  program/domain 
specific  problems. 

Cross  program  focus  demands  collaborative 
approach  and  shared  research  facilities;  high 
degree  of  experimentation  is  involved  on 
continuing  basis;  collaboration  of 
government,  industry  and  academia  is 
essential.  Research  directed  at  crosscutting 
problems  involving  complex  synergies  of 
technologies,  socio-technical  issues,  policy, 
and  enterprise  factors. 

PUBLIC  RELEASE 

30 


PUBLIC  RELEASE 


2.  Problem  Characterization 


Do  These  Systems  Work  Together? 

(*will  we  ever  need  them  to  do  so?)  %£# 


Consider  the  USAF  systems  appearing  on  the  chart:  Do  they  work  together?  What  about 
the  Army  and  Navy  systems,  can  we  interoperate  with  them?  What  about  the  systems  of  other 
nations,  can  we  interoperate  with  them  as  well?  (Note  that  “coalition”  is  not  limited  to  countries 
with  English  as  their  native  language.)  The  question  we  must  wrestle  with  is  “will  we  EVER 
need  them  to  work  together?  (If  so,  what  could  we  do  about  it?)” 
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Do  These  Systems  Work  Together? 
(*will  we  ever  need  them  to  do  so?) 


We  only  buy  them  to  work 
together  if  we  thought  of  it 
before  delivery... 


b.Dl 

□  CQA  fc^jACCEPTANCE  of  listed  items  has  been  made  by 
me  or  under  my  supervision  and  they  conform  to  contract,  except  as 
noted  herein  or  on  supporting  documents. 


SIGNATURE  OF  AUTHORIZED 
GOVERNMENT  REPRESENTATIVE 


‘The  sum  of  the  wisdom  is  a  cursor  over  the  target’ 


For  each  of  the  U.S.  systems  involved,  there  is  something  called  a  DD250.  This  is  the 
final  document  in  a  system’s  development  process.  It  is  the  one  where  the  “Authorized 
Government  Representative”  checks  the  box  and  signs  on  the  line,  accepting  the  system  and 
permitting  payment  to  the  contractor.  If  a  delivered  system  is  going  to  play  with  other  systems,  a 
developer  must  consider  that  inter-play  and  include  it  in  the  requirements  for  the  system.  Our 
traditional  Systems  Engineering  approach  has  excellent  provisions  for  supporting  those  “work 
together”  relationships  if  they  are  included  in  the  original  requirements.  However,  what  do  we 
do  about  unanticipated  CONOPS?  In  addition,  what  should  we  do  about  unanticipated  inter¬ 
system  relationships? 

If  the  Air  Force  is  to  realize  the  true  potential  of  “Sum  of  the  wisdom  is  cursor  over 
target,”  it  needs  to  build  systems  in  a  way  that  allows  them  to  behave  in  the  most  robust  of 
network  centric  concepts.  We  need  to  build  them  with  system-of-systems  potential  enabled  from 
the  beginning.  In  the  absence  of  having  built  systems  to  be  system-of-systems  configured,  we 
need  a  means  for  backfilling  systems. 

The  questions  remain.  What  enabling  technology  do  we  need  to  define?  How  will  we 
discriminate  good  technologies?  How  will  we  know  which  component  systems  we  should  outfit 
with  system-of-systems  enablement? 
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Today's  Systems  Do  Not  Interoperate  frSN&f. 

Well....  Causes  Problems  '%j£/J 


Data  not  passed  or  passed  in 
incomplete  form 

Sneaker  net  information  transfer 

Makes  warfighter’s  job  more  difficult 

■  Increased  workload  for 
workarounds 

■  Limited  situation  awareness 

■  Reduced  performance 


Who  is  in  the 

What  information 

network  now? 

can  1  get  from 

.  them?  > 

Misses  advantages  of  information 
technology 

■  Missed  target  opportunities 

■  Fratricides 

■  Slow  sensor  to  shooter 


Who  is  doing 
what  and  where 
are  they? 


How 

information/capability 
be  available? 


“Blackhawk  Down” 

—  A  really  bad  experience 


Today’s  systems  not  being  developed  with  a  system-of-systems  perspective  provide  only 
a  loosely  coupled  conglomeration.  They  either  do  not  pass  data  at  all,  or  only  partially  pass  data 
between  components.  Often  the  human  must  manually  derive,  transform,  and  reenter 
information  between  systems. 

As  the  option  of  last  resort,  the  human  operator  can  only  partially  compensate  for  these 
shortcomings.  In  trying  to  gather  and  process  the  needed  information,  he  or  she  ends  up  with  a 
significantly  increased  workload  and  limited  situation  awareness.  In  many  cases,  they  may  never 
discover  what  they  really  needed  to  know. 

Sometimes,  as  experiences  from  Somalia  would  indicate,  the  human  becomes  overloaded 
trying  to  compensate  for  situation  awareness  mismatches  among  supporting  systems.  There 
were  instances  where  people  were  talking  to  party  A,  when  they  thought  they  were  talking  to 
party  B.  The  consequences  when  Party  A  executed  instructions  intended  for  Party  B  were 
disastrous.  Likewise,  when  Party  B  never  got  their  intended  instructions,  the  outcome  was  not 
desirable. 

The  result  of  systems-of-systems  having  poor  interoperability  is  that  we  are  failing  to 
realize  the  true  potential  of  what  all  systems  working  together  as  a  whole  can  achieve.  This 
results  in  missed  target  opportunities,  fratricides,  and  slow  sensor  to  shooter  performance. 
Today’s  systems  do  not  operate  effectively  as  true  systems-of-systems. 
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3.  The  System-of-Systems  Engineering  Methodology 

To  address  these  shortcomings  of  the  current  state  of  affairs,  the  Air  Force  needs  a 
System-of-Systems  Engineering  process.  We  have  identified  four  main  factors  that  require 
consideration  in  improving  System-of-Systems  Engineering  in  the  Air  Force: 

•  The  first  of  these  is  the  need  to  include  human  system  interaction  as  a  part  of  System-of- 
Systems  Engineering 

•  The  second  is  the  need  to  identify  the  convergence  protocols  that  enable  the  critical  loose 
coupling  linkages  among  system  components 

•  The  third  is  the  need  to  address  the  key  motivators  in  the  system  procurement  and 
development  process  that  currently  hamper  good  system-of-systems  development 

•  The  last  is  the  need  to  incorporate  discovery  learning  through  experimentation  at  the 
system-of-systems  level 
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3.1.  The  Human  Role 


Human  System  Interaction  is  Critical 

in  System-of-Systems 


■  Non-interoperable  systems  heavily  load  the  human-human  link  to 
compensate  for  poor  system-system  links 

■  If  component  systems  have  poor  HSI,  the  situation  is  exacerbated 


■  Effective  systems  of 
systems  can  solve  this 
problem  by  optimizing  the 
information  flow  among 
systems 


■  Human-Human  and  Human- 
System  Interactions  will 
continue  to  be  important 

■  Need  a  minimal  set  of 
consistent  &  compatible 
human  interaction 
standards,  (e.g.  MIL  Std 
2525  Symbology,...) 


Common  Operational  Context 

ft 


Shared  SA 
Joint  Decision  Making 


Human 

ware 


Software 


ft 


Consistency 

Compatibility 

Open  Architectures 
Standard  Data  Formats 
Service  Based 
Common  Data  Sharing 
Discovery 
Interoperability 
Commonality 


Software 


Discovery 

Interoperability 

l  JM 


In  an  effectively  performing  system-of-systems,  mission  performance  is  dependant  on 
effective  joint  decision-making  and  collaboration  among  the  components.  Achieving  this  is  all 
about  enabling  effective  and  efficient  information  sharing. 

In  today’s  systems,  where  direct  sharing  of  information  between  systems  is  low,  the 
human-to-human  link  is  heavily  loaded  in  shouldering  this  burden. 

Systems-of-systems  can  address  this  problem  by  significantly  improving  the  information 
flow  between  the  systems  components.  We  acknowledge  this  network  connectivity  as  a  key 
component  of  a  system-of-systems. 

However,  we  should  recognize  that  human-to-human  and  human-to-system  interactions 
would  continue  to  be  critical  components  of  effective  systems-of-systems.  Collaborative 
decision  making,  for  example,  requires  that  human  operators  share  a  common  picture  of  what  is 
happening  on  the  shared,  relevant  aspects  of  the  mission.  Just  as  systems  need  to  be  able  to 
interoperate  in  order  for  people  to  interoperate,  they  also  need  an  established  set  of  commonality 
and  consistency  rules  in  their  interface  designs.  These  rules  need  to  be  addressed  within  System- 
of-Systems  Engineering.  Regardless  of  the  system-to-system  collaboration  requirements,  the 
individual  systems  need  to  incorporate  sound  human-to- systems  interfaces. 
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SoS  &  the  Info  Sharing  Model 


Dedicated  Networked 


Networked  Broadcast 


Links 

Point  to  Point 
connections 
dedicated 
comms, 

unique  msgs,  etc 


Links 

Information 
shared  among 
“known 

needy  friends” 
Ex.  DCGS,  FCS 


Links  with 
Discovery 
Technology 

Sharing  among 
“unknown 
needy  friends” 


Information 
broadcast 
to  everyone 
«=^  Overload 


The  recognition  that  the  systems  supporting  warfighters  could  perform  considerably 
better  than  they  have  if  users  could  more  easily  interconnect  them  partially  drives  the  emergence 
of  network  centric  operations.  Interoperability  among  systems  is  complex.  Of  course,  they  need 
to  have  communications  connectivity.  In  earlier  days,  the  connectivity  among  systems  was 
established  with  unique  communications  devices.  The  flexibility  was  very  limited,  since  only 
systems  equipped  with  the  particular  communications  mechanisms  could  pass  information 
among  each  other. 

The  emergence  of  the  network  allows  a  simplified  communications  structure:  Presence 
on  the  network  allows  (in  principle)  information  access.  In  practice,  we  build  systems  to  allow 
information  exchange  via  “internet”  standards,  but  unless  we  establish  prior  arrangements,  the 
benefits  of  network  enablement  are  limited  to  the  exchange  of  bits  rather  than  the  exchange  of 
“information.”  Systems’  internal  business  rules  for  what  information  is  accessed  and  how  it  is 
processed  are  still  structured  upon  the  presumption  that  a  definition  exists  for  “information 
exchange  requirements”  for  every  information  exchange  over  the  network. 

One  form  of  connectivity  over  a  network  would  allow  broadcast  of  information  to  each 
recipient  system.  Not  only  does  this  challenge  the  bandwidth  capabilities  of  communications 
systems  that  support  the  net,  but  also  the  resultant  information  overload  can  diminish  rather  than 
enhance  the  operator’s  responses. 

A  healthy  balance  between  information  overload  and  information  exchange  among 
predetermined  recipients  would  be  the  emergence  of  “discovery  agents”  or  “brokers”  which 
recognize  the  opportunity  for  recipients  to  receive  information  that  they  might  otherwise  miss. 
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SoS  &  the  Info  Sharing  Model 


Connectivity 

Effectiveness 


Dedicated 

Links 

Point  to  Point 
connections 
dedicated 
comms, 


O 

Networked 

Links 

information 
shared  among 
“known 


4^ 

Networked 
Links  with 

Discovery 

Technology 

Sharing  among 


ideas  ( e.g .,  “broker”)  from  JBI  studies  are  just 
barely  starting  to  take  root.., 
we're  way  late  in  technology  development 


Broadcast 

information 
broadcast 
to  everyone 
«=£> Overload 


The  notion  of  a  broker  is  not  new.  The  developers  of  the  Joint  Battlespace  Infosphere 
concept  built  it  upon  the  premise  that  they  could  introduce  this  technology  into  the  network. 
Sadly,  the  emergence  of  this  technology  is  not  taking  place  in  military  systems.  Some 
commercial  development  has  taken  place.  This  development  continues  to  gain  interest  in  the 
form  of  sophisticated  search  engine  techniques.  However,  these  search  engines  do  not  have  the 
real-time  performance,  secure  authentication,  or  commander’s  intent  properties  that  a  discovery 
agent  should  show  in  the  DoD  network. 

There  is  a  need  for  technical  investment  into  this  technology  that  is  critical  to  the 
achievement  of  the  network  centric  operations  vision. 
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3.2.  Convergence  Protocols 

Systems-of-systems  are  all  around  us.  Some  examples  are:  The  highway  system,  the 
various  utilities,  the  internet,  the  high-level  architecture  protocol  maintained  by  the  Institute  of 
Electrical  &  Electronic  Engineers  (IEEE  1516)  which  allows  simulation  models  to  work 
together,  and  so  forth.  Consider  the  collection  of  electrical  appliances  in  a  house.  A  complex 
collection  of  power  plants,  hydroelectric  dams,  solar,  and  wind  driven  generators  provide 
electricity.  It  does  not  matter  which  system  provides  electricity  for  a  particular  household 
appliance.  The  systems-of-systems  that  generate  electricity  all  are  able  to  provide  that  power  to 
a  diverse  set  of  appliances  in  a  household  through  the  common  “loose  coupling”  devices  known 
as  wall  sockets  and  plugs.  Note  that  a  mechanical  standard  is  insufficient  to  allow  these  systems 
to  interconnect.  One  needs  to  also  specify  voltage  (110,  or  thereabouts),  60  Hz  (and  accurately 
maintained  at  60  Hz  or  the  electric  clocks  will  not  keep  proper  time.)  The  collection  of  physical 
and  electrical  properties  comprises  a  “protocol.” 

Protocol  is  a  term  used  to  cover  a  variety  of  properties.  For  information  systems,  a 
protocol  would  characterize  not  only  the  physical  connector,  but  it  may  also  refer  to  the  value  of 
voltage,  frequency  of  a  signal,  pulse  widths,  and  so  forth  that  are  used  to  convey  information. 
The  protocol  definition  may  also  characterize  the  sequence  of  bits  (e.g.,  most  significant  bit  first 
or  most  significant  bit  last  in  a  series  of  bits).  It  may  include  information  field  definitions  so  that 
the  system  passes  different  predefined  parameters  as  part  of  the  handshake  between  two  systems 
using  a  protocol.  In  other  words,  the  protocol  can  be  so  complex  that  it  serves  many  information 
exchange  issues  that  commonly  occur  between  information  systems.  For  non-information 
systems,  protocols  can  likewise  be  relatively  complicated.  For  example,  in  the  highway  system, 
the  protocol  for  traffic  management  includes  rules  (such  as  drive  on  the  right,  stop  for  red  lights, 
and  so  forth)  as  well  as  physical  properties  associated  with  the  width  of  roadways,  permitted 
weights  and  heights  of  vehicles,  and  so  forth. 

In  all  cases,  the  use  of  protocols  defines  certain  expected  behaviors  among  the 
participants  in  a  system-of-systems. 

Sometimes,  a  protocol  emerges  that,  by  influence  of  common  acceptance,  becomes  an 
enabling  protocol  for  market  access.  Conformance  to  the  common  protocol  provides  benefits  to 
the  users  of  the  protocol  because  it  allows  them  to  leverage  the  number  of  other  conforming 
products  to  improve  further  their  product’s  attractiveness  among  users.  It  would  not  be  very 
attractive  for  a  vendor  of  lights  to  “invent”  a  new/better  light  bulb  that  has  a  socket  that  is 
different  from  what  is  commonly  found  among  lamps.  In  exceptional  cases,  vendors  may 
attempt  divergence  from  the  standard,  but  the  investment  is  high,  and  the  penalty  for  non- 
acceptance  is  market  rejection.  Hence,  most  purveyors  of  light  bulbs  manufacture  them  to  fit  in 
a  standard  socket  and  to  operate  with  120V,  60Hz,  AC. 

When  a  community  popularly  accepts  a  single  protocol,  and  this  protocol  allows  multiple 
systems  to  interact  with  one  another,  we  call  that  protocol  a  “Convergence  Protocol,”  because  the 
community  has  converged  upon  it  as  a  universal  standard. 


PUBLIC  RELEASE 
41 


PUBLIC  RELEASE 


Key  Element:  Layered  Models 


■  A  hierarchical  model  of  a  system 
describes  it  in  terms  of  subsystems, 
components,  etc. 

■  Interfaces  link  components.  The 
interfaces  might  themselves  be 
thought  of  as  components 

■  An  alternative  way  of  viewing  a  system- 
of-systems  is  in  layers  -  each  layer 
expresses  “services”  which  upper 
layers  utilize 

■  A  (unanticipated?)  by-product  of 
layering  has  been  the  adoption  of  pre¬ 
eminent  layers  (e.gv  IP  in  the  case  of 
network  protocols). 

■  These  are  layers  which  become 
“market-enabling,  convergence 
protocols” 


In  traditional  Systems  Engineering,  a  complex  system  is  broken  down  into  a  nested 
hierarchy  of  subsystems,  with  interfaces  linking  the  individual  components.  This  point  of  view 
works  very  well  for  systems  with  top-level  requirements  and  a  single  entity  responsible  for 
product  development.  However,  it  is  difficult  to  implement  this  for  systems-of-systems,  because 
the  requirements  are  often  not  known  when  the  individual  systems  are  being  designed  and  they 
change  as  the  systems  (and  operational  uses  of  the  systems)  evolve. 

A  key  approach  to  System-of-Systems  Engineering  in  the  commercial  sector,  especially 
in  the  communications  industry,  is  the  use  of  “layered  architectures.”  In  a  layered  model,  the 
overall  system-of-systems  is  broken  down  into  different  collections  of  services,  with  each 
collection  expressing  the  services  that  are  available  to  layers  above  it  in  the  “protocol  stack.” 
Layered  architectures  allow  different  developers  to  work  in  parallel  and  insure  that  changes  in 
one  layer  of  the  protocol  do  not  interfere  with  operations  above  and  below  that  layer.  Thus,  a 
layered  protocol  implements  loose  coupling  between  the  services  that  makes  up  the  overall 
system-of-systems. 
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Internet  as  a  System-of-Systems 


•  Probably  the  most  complex 
engineered  system  in  existence 

■  The  architecture  of  the  Internet  is 
the  Internet  Protocol  (IP),  not  the 
physical  structure,  and  not  a  set  of 
cartoon  diagrams 

■  The  architecture  is  no  less 
real  for  not  being  physical 

■  IP  defines  packet  structure, 
addressing,  routing 

■  Collaborative  bodies  control 
architecture  evolution 

■  IETF,  etc. 

■  Socio-technical  structural 
synergy 

■  Structure  repeats  in  the  web 


The  prototypical  example  of  a  system-of-systems  is  the  Internet,  which  is  perhaps  the 
most  complex  engineered  system  in  current  existence.  The  Open  Systems  Integration  (OSI) 
Reference  Model  specifies  the  protocol  stack  for  the  Internet.  This  model  specifies  the  services 
provided  at  seven  layers  (i.e.,  application,  presentation,  session,  transport,  network,  data  link,  and 
physical).  Examples  of  application  layer  protocols  include  HyperText  Transport  Protocol 
(HTTP)  and  File  Transfer  Protocol  (FTP).  Examples  of  network  protocols  include  IP  (Internet 
Protocol),  NetBIOS  Extended  User  Interface  (NetBEUI),  and  X.25.  IP  has  emerged  as  a  clear 
standard  at  the  network  layer;  it  defines  the  structure  of  a  packet  of  information,  including 
information  required  for  addressing  and  routing.  Indeed,  IP  has  become  so  ubiquitous  it  has 
enabled  entire  industries  by  making  their  products  IP  compatible. 

IP  is  thus  an  example  of  a  “convergence  protocol.”  Convergence  protocols  enable 
systems-of-systems  by  creating  the  opportunity  for  rapid  and  diverse  innovation  both  above  and 
below  the  convergence  protocol.  By  creating  a  single  standard  that  everyone  can  use,  developers 
at  lower  layers  of  the  protocol  can  innovate  without  worrying  about  the  applications  that  will 
eventually  make  use  of  their  components  (assuming  that  they  use  the  convergence  protocol). 
Similarly,  developers  at  higher  levels  of  the  protocol  stack  do  not  need  to  concern  themselves 
with  the  lower  level  details  of  the  system  structure. 
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Derivation  of  Convergence  fsi 

Protocols 


■  What  stimulates  creation  of  Convergence  Protocols? 

■  What  technology  enables  us  to  make  them? 

■  What  Systems  Engineering  build  &  management 
processes  provide  good  results? 


Type  A  -  Monopolies 

■  Single  entity  controls  all  decisions  (and  $$) 

■  Example:  RJ11  phone  jacks  . 

■  Dictated  by  AT&T,  which  owned  everything 

■  Became  a  worldwide  standard  (almost) 

Type  B  -  Free  markets 


■  Governed  by  (emergent)  community  standards 

■  Example:  H.323  video  format 

■  Set  up  by  Int’l  Telecommunications  Union  (ITU) 


■  Enables  desktop  conferencing  (e.g.,  NetMeeting) 


As  discussed  in  the  Domain  Integration  study,  systems  can  connect  to  one  another  in  a 
variety  of  ways.  When  systems  have  tight  interdependencies,  the  connections  often  intertwine 
with  fundamental  behaviors  inside  each  system  that  is  being  brought  together.  The  so-called 
“tight  coupling”  implies  detailed  knowledge  about  each  system  and  well-coordinated  interfaces, 
which  have  many  and  complex  ways  of  interacting  to  support  the  system- to- system 
interconnections.  On  the  other  hand,  “loosely  coupled”  systems  implies  that  there  are  some 
commonly  understood  rules  of  engagement  that  allow  the  systems  to  pass  information  without 
needing  intimate  knowledge  about  the  internal  operations  of  each  system.  These  desirable  loose 
couplings  allow  much  flexibility  among  systems  that  are  interoperating. 

Several  features  of  convergence  protocols  help  define  this  important  class  of  loose 
couplers.  First,  they  must  enable  independent  evolution  of  technology  on  each  side  of  the 
protocol.  Second,  they  are  typically  long-lived.  Since  many  people  build  on  them,  they  hence 
become  essential  elements  over  many  generations  of  technological  change.  In  the  case  of  the 
Internet,  IP,  which  was  first  implemented  in  the  1970s,  remains  the  dominant  network  protocol 
in  use  today;  connecting  billions  of  devices  of  staggering  variety. 

Convergence  protocols  (and  standards  in  general)  can  be  developed  in  many  ways.  One 
common  structure  is  the  existence  of  a  single  entity  that  has  control  over  the  entire  collection  of 
technologies  required  to  develop  a  given  capability.  A  commercial  example  of  this  is  the 
development  of  the  RJ11  phone  jack,  which  is  the  standard  connection  for  telephones  (and 
modems)  in  the  US  and  most  of  the  world.  AT&T  developed  RJ11  and  dictated  this  standard 
because  it  controlled  the  telephone  market  at  the  time.  A  second  method  by  which  convergence 
protocols  and  standards  are  developed  is  through  free  markets.  This  approach  typically  involves 
groups  of  companies,  standards  organizations,  and  other  communities  of  interest  coming  together 
to  find  a  mutually  beneficial  protocol.  Often  several  different  protocols  emerge  (Beta  versus 
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Vertical  Helical  Scan  or  VHS),  and  the  market  determines  whether  multiple  protocols  survive  or 
whether  one  becomes  the  dominant  protocol  (VHS,  IP,  and  so  forth). 
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Commercial  vs  DoD  Markets 


■  Commercial  systems  of  systems  rely  on  future 
profit  as  incentive  =>  incorporate  standards 

■  Consumers  buy  systems  that  can  interact 
with  other  systems  over  time 

■  Example:  extra  laptop  features  (USB,  SVGA, 
802.1  lx,  Bluetooth) 

■  DoD  pays  for  development  of  technology  and 
cuts  funding  =>  develop  specialized  interfaces 


Little  incentiygje-metud^standard  interfaces 
if  they  don’  enable  KPPs 
Flexibility  gets  fiui  uufflig  budget  crises 
DoD  will  to  pay  to  fix  the  problem  later 


IN/OUT 


SoS  for  DoD  requires  different  approach 


An  important  finding  of  the  study  is  that  the  techniques  for  building  an  effective  system- 
of-systems  in  the  commercial  world  are  not  likely  to  work  in  the  DoD.  In  commercial  systems 
future  profit  provides  an  incentive  for  seeking  and  using  convergence  protocols.  Hence, 
additional  features  may  be  included  on  a  product  because  it  will  enable  the  system  to  become 
part  of  a  system-of-systems  later.  In  the  DoD  there  is  little  incentive  for  including  standard 
interfaces  if  they  do  not  enable  key  performance  parameters  (KPPs).  Although  integration  of 
standard  interfaces  is  often  present  at  the  beginning  of  a  program,  program  managers  can  cut 
features  that  add  cost  to  the  product  later  in  the  program  as  budget  issues  arise.  The  DoD 
exacerbates  this  situation  by  its  willingness  to  pay  later  to  fix  the  problem;  when  it  would  have 
been  cheaper  to  address  the  problem  by  including  the  standard  interface.  Thus,  we  believe  that 
enabling  SoSE  in  the  DoD  will  require  a  different  approach. 
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If  Convergent  Protocols  f5»l 
Were  Available... 


•  Joint  Unmanned  Combat  Air  Systems  (J-UCAS) 

■  Currently  developing  a  common  operating 
system  and  demonstrator  air  vehicles 

■  The  development  and  evolution  of 
requirements  is  stated  as  a  critical  element  of 
the  program 

■  Emphasis  is  being  placed  on  insuring  intra¬ 
operability  between  planned  and  potential 
internal  components 

■  Similarly,  emphasis  is  being  placed  to  insure 
inter-operability  with  external  elements  such  as 
manned  aircraft,  C2  centers,  space  assets,  etc. 

■  Would  convergence  protocols  allow  for 
increased  flexibility  and  adaptability? 


■  i.e.,  better  system  of  system  inter¬ 
operability  and  enhanced  future 
capabilities 


As  described  previously,  convergent  protocols  are  those  protocols  that  allow  systems  to 
interact  and  communicate  at  key  points  across  system  boundaries.  Imagine  for  a  moment,  if 
these  protocols  were  available  for  the  Joint  Unmanned  Combat  Air  Systems  (J-UCAS)  program. 
How  would  the  existence  of  these  protocols  ease  the  programs  burdens  and  provide  for  better 
system-of-systems  operability? 

The  J-UCAS  is  currently  developing  a  common  operating  system  (COS)  and  two 
different  air  vehicles  variants,  the  X-45  and  X-47.  The  X-45  variant  focuses  on  meeting  Air 
Force  mission  requirements,  while  the  X-47  variant  addresses  Naval  missions.  One  of  the  main 
objectives  of  the  program  is  the  development  and  evolution  of  system  level  requirements.  The 
systems  and  programs,  which  are  a  result  of  J-UCAS  initiatives,  will  need  to  have  elements  of 
both  intra-  and  inter-operability. 

Well-defined  system  requirements  and  proper  Systems  Engineering  will  result  in  systems 
designed  with  a  high  degree  of  intra-system  capability.  This  intra-system  capability  will  result  in 
system  elements,  such  as  radar  sensors,  electronic  warfare  components,  and  ground  elements, 
which  developers  and  users  can  interchange  and  upgrade  to  meet  changing  mission  requirements. 
Additionally,  this  intra-operability  greatly  enhances  the  spiral  development  process,  which  is 
common  in  the  current  acquisition  process. 

Developing  inter-operable  systems  pose  another  set  of  challenges.  This  is  especially  true 
when  future  needs  are  uncertain  and  it  is  unknown  what  specific  systems  will  need  to  work 
together  as  a  system-of-systems.  If  convergent  protocols  existed,  it  is  likely  that  it  would  be 
easier  to  develop  systems  with  the  flexibility  and  adaptability  to  meet  true  system-of-systems 
inter-operability. 


PUBLIC  RELEASE 
47 


PUBLIC  RELEASE 


(This  page  intentionally  left  blank.) 
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3.3.  Motivations 


Motivators 


■  Lots  of  guidance  exists  on  what  should  be  done 


But,  no  specifics  exists  on  how  to  do  it... 


Next,  we  must  explore  the  motivations  for  the  participants  in  a  system-of-systems 
development  so  that  the  Air  Force  can  incentivize  behavior  towards  cultivating  more  systems 
enabled  for  participation  in  systems-of-systems. 

The  traditional  approach  for  the  DoD  to  encourage  adoption  of  commonality  among 
systems  is  to  issue  a  directive,  instruction,  or  guidance,  which  forces  program  managers  to 
conform.  There  is  much  guidance  in  the  case  of  network  centricity.  The  two  small  diagrams 
above  include  a  calendarized  list  of  documents  released  that  declare  the  vision  and  provide 
guidance  for  program  managers  to  adopt  the  Global  Information  Grid  (GIG).  It  is  healthy  and 
good  to  describe  the  intended  goals  for  achieving  the  benefits  of  network  centricity. 

However,  the  provided  guidance  is  insufficient.  The  instructions  are  too  vague  for  a 
program  manager  or  contractor  to  incorporate  with  a  successful  outcome  that  transcends  the 
Responsibility,  Authority,  Accountability,  and  Resources  (RAAR)  limits  of  his  or  her  project. 
There  is  a  need  for  a  single  entity  to  declare  sufficient  specificity  among  protocol  candidate 
standards  that  everyone  could  apply  the  protocol  as  a  Market  Enabling  Convergence  Protocol. 

The  traditional  DoD  approach,  to  declare  a  solution  and  require  everyone  to  conform, 
may  surface  with  respect  to  convergence  protocols  prematurely.  When  this  has  happened  in  the 
past,  the  consequences  have  not  been  good.  It  is  important  not  to  declare  guidance  and  directives 
for  an  unproven  technology.  Hence,  we  propose  a  different  approach  for  the  SoSE  methodology. 
First,  establish  hands-on  experience  with  the  appropriate  convergence  protocols.  Then,  issue 
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such  guidance  as  appropriate  for  program  managers.  Do  not  do  this  in  the  opposite  sequence 
(which  appears  to  be  the  current  path) 

Note  also,  our  suggestions  are  not  about  adopting  a  specific  vendor’s  “product;”  the 
methodology  focuses  upon  a  derived  protocol  implementation  standard. 
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Motivators 


■  Motivators  are  needed  to  encourage  users  to  ask  for, 
acquirers  to  procure  and  developers  to  deliver  systems 
capable  of  working  in  an  interoperable  and  adaptive 
systems  of  systems  environment 

■  Motivators  are  different  yet  have  a  common  basis  for 
each  group 

■  Users  -  Desire  to  obtain  maximum  capability 

■  Acquirers  -  Desire  to  meet  user  requirements 

■  Developers  -  Desire  to  meet  short  and  long  term  business 
goals,  which  can  be  tied  to  meeting  user  requirements 

■  What  is  needed  is  a  way  to  determine  the  System  of 
System  derived  requirements 


Given  that  commercial  market  forces  are  inappropriate  for  the  DoD,  some  alternative 
motivational  approach  is  necessary.  Contrary  to  commercial  practice  where  the  developer 
speculatively  invests  in  something,  perhaps  with  total  ignorance  on  the  part  of  the  user;  the  DoD 
user  needs  to  express  requirements  for  something.  With  regard  to  convergence  protocols,  neither 
program  managers  nor  contractors  can  include  them  based  on  anticipated  future  needs;  only  the 
user  can  legitimately  express  the  requirement. 

Therefore,  the  SoSE  strategy  is  to  discover  motivators  for  use  within  the  DoD. 
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Acquisition  Program  Requirements 


Ref:  Mike  Kerns-  OSD/NII 


In  the  case  of  the  GIG,  there  is  an  effort  underway  to  express  more  specificity  in  terms  of 
what  the  individual  programs  need  to  incorporate  to  be  able  to  achieve  system-of-systems 
interaction  via  the  GIG.  However,  there  is  a  “3-bears”  problem:  Too  much  detail  and  DoD  over 
constrains  systems,  not  enough  detail,  and  the  objective  behavior  is  not  met.  Finding  the  “just 
right”  level  of  detail  can  only  be  derived  via  experimentation.  Unfortunately,  the  authors  of 
these  policy  documents  do  not  (yet)  have  the  experimental  laboratory  or  venue  necessary  to 
discover  the  right  answer. 

In  other  words,  it  is  necessary  to  re-emphasize  the  cautionary  note  expressed  earlier  about 
premature  declaration  of  standards.  The  people  promoting  more  specific  program  manager 
instructions  do  understand  the  challenges  faced  by  program  managers  with  vague  guidance; 
however,  premature  specific  guidance  could  be  harmful. 

So,  although  this  section  began  with  somewhat  disparaging  remarks  towards  the  volume 
of  documented  guidance,  instructions,  advice,  and  so  forth,  it  ends  with  an  endorsement  of  just 
that  kind  of  activity  (albeit  with  one  substantial  modification:  Emphasis  on  experience  bom  of 
experimentation).  This  leads  us  into  the  next  and  final  discussion  topic:  Experimentation. 
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3.4.  Experimentation 


Three  Experimentation  Venues 


Legacy  Products 
(Live  &  Virtual) 


Seven  Stakeholders” 


The  commercial  marketplace  is  an  active  environment  in  which  one  can  continually  test 
and  evaluate  emergent  system-of-systems  opportunities.  On  the  other  hand,  the  environment  is 
mostly  asleep  (hopefully)  for  DoD  products  because  war  is  never  a  preferred  condition  of 
operation.  Nevertheless,  it  is  during  wartime  when  the  most  dramatic  alterations  to  DoD  (USAF) 
CONOPS  take  place.  This  suggests  that  there  is  a  need  for  a  “pseudo-war”  experimentation 
venue  to  substitute  for  the  active  commercial  market  environment  that  drives  commercial 
systems-of-systems.  The  absence  of  such  an  experimentation  venue  seems  to  cause  stagnation  of 
USAF  capability  aspirations  around  the  achievements  of  the  “last  fought  war.”  Disruption  of  the 
status  quo  is  an  important  ingredient  in  system-of-systems  operation.  Therefore,  if  the  Air  Force 
is  to  achieve  the  purposes  of  this  study,  it  must  find  effective  methods  of  disrupting  the  status 
quo. 

With  rare  exception,  most  DoD  SoS  development  has  been  initiated  as  “on  the  fly” 
intersystem  couplings,  enabling  CONOPS  that  meet  immediate  wartime  operational  needs. 
While  there  is  a  plethora  of  DoD  documentation  espousing  the  value  and  need  for  SoS,  DoD  has 
mostly  been  unsuccessful  in  generating  strong  motivation  to  identify  and  develop  SoS  enablers 
during  peacetime.  Ingrained  acquisition  approaches  and  conventional  SE  methodology  have  not 
encouraged  users,  acquirers,  and  developers  to  adopt  an  SoS  perspective  in  the  design, 
development,  and  use  of  DoD  systems. 

A  key  recommendation  of  this  study  is  to  use  robust  experimentation  to  show  the  value  of 
enabling  SoS  participation  to  system  level  users,  acquirers,  and  developers.  This 
experimentation  would  consist  of  three  distinct  venues: 
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Concept  Experimentation  -  a  mechanism  for  low-cost,  low-entry  barrier 
experimentation,  with  low  consequence  of  failure,  for  developing  new  CONOPS  for 
collections  of  existing  systems  and  evaluating  proposed  systems  and  their  impact  on 
existing  systems. 

Intersystem  Coupling  Experimentation  -  a  mechanism  for  ongoing  development  of 
intersystem  couplers;  identifying  those  that  by  their  utility  become  convergence  points 
that  must  be  considered  in  the  development  of  all  future  DoD  systems. 

Provisional  Fielding  -  a  mechanism  for  rapid  experimental  fielding  of  an  emerging  SoS 
concept  that  provides  an  initial  capability,  crystallizes  user(s)  requirements,  and  focuses 
the  spiral  development  of  the  SoS  component  systems. 
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Figure  1 :  Three  experimental  venues  for  promoting  SoS  development 


Figure  1  presents  an  SoS  experimentation  flow  diagram  illustrating  for  each  of  the  three 
venues  the  kinds  of  efforts  undertaken  in  each  venue,  the  output  products  of  each  venue,  and 
interconnections  of  efforts  and  output  products  for  each  venue.  All  three  would  occur  in  parallel, 
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providing  feedback  to  one  another  to  create  an  ongoing  experimental  “Petri  dish”  environment 
for  SoS  development. 

This  structure  allows  for  a  division  of  responsibility  among  IJSAF  organizations, 
focusing  attention  and  leadership  in  addition  to  better  coupling  research,  acquisition,  and 
operator  interests  early  in  the  process  of  developing  complex  SoS. 

For  example,  concept  experimentation  and  SoS  CONOPS  development  might  best  be 
“owned”  by  operators  such  as  AF/XO  or  Air  Combat  Command  (ACC).  Intersystem  Coupling 
Experimentation  is  of  central  value  to  organizations  with  integration  responsibilities  such  as 
SAF/XC  or  because  of  its  largely  generic  R&D  character  to  Air  Force  Research  Laboratory 
(AFRL).  Provisional  fielding  might  be  the  purview  of  SAF/AQ  as  the  lead  office. 


Concept  Experimentation 

The  first  experimental  venue  for  SoS  exploration  is  that  of  Concept  Experimentation. 
This  venue  would  focus  on  exploring  SoS  architectures,  developing  SoS  CONOPS,  evolving  the 
Tactics,  Techniques,  and  Procedures  (TTPs)  of  constituent  systems,  and  experimenting  with 
intersystem  interactions  to  define  SoS  concepts  that  show  beneficial  emergent  capabilities  and 
operational  utility  beyond  the  “sum  of  the  parts.”  Making  this  a  relatively  low-cost  and  largely 
analytical  exploration  will  provide  a  low  entry  threshold  mechanism  that  the  Air  Force  can 
promulgate  as  a  continuous  (i.e.,  peacetime)  exercise  and  evolve  into  an  expectation  for: 

•  Users  to  investigate  new  CONOPS  and  explore  SoS  solutions  to  unmet  or  emerging 
needs; 

•  Acquirers  to  establish  SoS  communities  of  interest,  i.e.,  which  systems  need  to  be 
procured  with  common  downstream  SoS  convergence  points  (“hooks”)  as  an  integral  part 
of  individual  system  developments;  and 

•  Developers  such  as  the  Defense  Advanced  Research  Projects  Agency  (DARPA),  AFRL, 
and  Contractors  to  identify  and  promote  new  mission  concepts,  discover  fruitful 
evolution  paths  for  existing  systems,  and  test  the  efficacy  of  new  systems  in  an  SoS 
environment. 

An  essential  element  of  concept  experimentation  is  making  it  widely  available  to  a  broad 
spectrum  of  potential  users  through  net-enabled  access  and  low  initial  cost  to  play.  This  will 
expand  the  potential  sources  of  SoS  ideas  beyond  users  in  the  field  inventing  “on-the-fly”  to 
meet  immediate  wartime  operational  needs.  Concept  Experimentation  will  enable  a  broader 
exploration  of  SoS  possibilities  independent  of  the  system  specific  trades  that  drive  current 
system  implementations  by  allowing  experimenters  to  readily  adjust  the  capabilities  and 
CONOPS  of  existing  systems  through  virtual  re-optimization,  and  providing  a  means  to  couple 
in  new  systems  without  having  to  hardwire  the  couplings  into  existing  systems. 

Our  recommendation  is  to  extend  current  modeling  and  simulation  capability  to  develop 
an  SoS  simulation  environment  that  can  bring  together  models  of  existing  systems,  simulations 
of  proposed  systems,  live-fly  exercises,  and  embodiments  of  adversary  CONOPS;  modify  them 
to  incorporate  SoS  enabling  intersystem  couplings;  and  evaluate  resulting  utility  against 
identified  user  needs  or  new  mission  capability.  For  example,  such  an  experimental  environment 
could  exploit  existing  networked  modeling  and  simulation  capabilities  (e.g.,  DMOC,  CWIN, 
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NCOIC  facilities,  and  so  forth).  These  facilities  would  make  available  models,  or  live  versions, 
of  existing  systems  for  experimentation.  The  experiments  would  allow  re-optimization  of 
individual  systems  and  intersystem  coupling,  unconstrained  by  tradeoff  limitations  that  would 
normally  affect  individual  systems  in  the  real  world. 

In  our  vision  an  experimenter  would  have  the  equivalent  of  “pull  down”  menus  for 
selecting  component  systems,  intersystem  couplers,  adversary  tactics,  and  basic  mission  structure 
(e.g.,  global  strike  and  so  forth).  These  can  be  “dragged  and  dropped”  into  a  software 
environment  through  a  graphic  user  interface  that  allows  access  to  each  component  model  for 
modification  of  system  capabilities,  CONOPS,  and  interfaces. 


Adversary  Tactics 


Couplers 


Systems  v 

• 

TSAT 

• 

F-22 

• 

J-UCAB 

• 

SBR 

• 

New  System 

Live  Fly  Exercise  Input 

if  Available 

Figure  2:  A  virtual  software  wrapper  for  SoS  concept  experimentation 


Different  combinations  of  systems,  using  different  intersystem  couplers,  flying  missions 
against  different  adversaries  can  be  implemented  virtually  in  search  of  SoS  emergent  behavior. 

A  key  product  of  concept  experimentation  is  the  identification  of  beneficial  emergent 
behavior;  the  “why”  that  drives  the  SoS.  Because  SoS  enabled  systems  invariably  will  incur 
additional  costs,  this  set  of  emergent  capabilities  provides  the  key  motivator  for  users  to  require 
the  incorporation  of  SoS  enabling  hooks  in  the  spiral  development  of  each  component  system. 

When  an  SoS  concept  is  shown  to  have  beneficial  emergent  behavior,  the  underlying 
question  changes  in  nature  to:  “what’s  missing?”  Whether  it  is  specific  intersystem  couplers, 
missing  systems,  or  champions  from  within  the  Community  of  Interest  (Col)  defined  by  the 
constituent  systems,  this  leads  to  discussions  with  Col  developers  on  impacts  of  proposed 
couplers  as  well  as  investigations  of  any  SoS  concept  specific  couplers  that  need  to  be 
developed. 
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Intersystem  Coupling  Experimentation 

The  next  experimental  venue  for  SoS  exploration  is  Intersystem  Coupling 
Experimentation.  An  important  element  of  experimentation  is  the  development  of  integrating 
tools  and  hooks  that  enable  intersystem  interactions.  These  intersystem  couplers  provide  a 
starting  basis  for  possible  intersystem  interactions  during  concept  experimentation.  They  are 
also  the  “plug-ins”  that  must  be  accommodated  for  rapid  provisional  fielding  experiments. 

Much  of  the  experimentation  in  this  venue  is  accomplished  independently  of  the  other 
two  venues  in  the  sense  that  it  seeks  to  mimic  the  successful  emergence  of  commercial  SoS  by 
identifying  and  developing  durable  coupling  protocols  and  standards.  Examples  include  data 
transfer  protocols,  communication  waveforms,  processing  algorithms,  logistics  interfaces,  system 
health  statusing,  capabilities  broadcast,  and  a  slew  of  network  issues  such  as  multi-level  security 
and  bandwidth  management. 

Experimentation  in  this  venue  will  examine  existing  standards  and  protocols.  This 
includes  those  in  development  as  commercial  standards  or  those  that  have  already  shown 
significant  utility  through  widespread  commercial  implementation.  Other  intersystem  couplings, 
which  might  be  unique  to  DoD  SoS,  are  also  examined  and  developed  as  part  of  this  effort. 
These  two  experimental  paths  generate  a  set  of  intersystem  couplers  that  can  be  used  as  the  tool 
set  in  Concept  Experimentation  (the  “couplers”  pull-down  menu). 


/\  Convergence  Protocol  -  a  ubiquitous  intersystem  coupler 
/\  SoS  Intersystem  Coupler 


Figure  3:  A  Depiction  of  Three  Intersystem  Couplers 
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Figure  3  illustrates  three  classes  of  intersystem  couplers  available  for  development  as  part 
of  this  research  thrust.  There  are  intersystem  couplers  that  are  ubiquitous.  Their  utility  and 
applicability  are  widespread  in  order  to  become  standards  that  the  Air  Force  must  consider  for  all 
on-going  system  development,  whether  it  is  for  new  systems  or  spiral  developments  of  existing 
systems.  Examples  include  IP,  High  Level  Architecture  (HLA)  for  distributed  computer 
simulation  systems,  and  automobile  hand  controls.  Other  intersystem  couplers  exist  that  fall 
short  of  being  a  convergence  protocol.  However,  these  intersystem  couplers  still  enable  a  system 
to  participate  in  an  SoS. 

A  third  and  important  class  of  intersystem  couplers  are  actually  isolators;  they  serve  to 
prevent  a  system  from  participating  in  a  particular  SoS.  An  example  of  an  isolator  is  the  narrow 
gasoline  fill  pipe  for  use  with  unleaded  gasoline  -  the  fill  pipe  prevents  introduction  of  leaded 
fuel  that  would  destroy  the  automobile’s  catalytic  converter.  Another  example  is  electrical 
outlets;  the  plug  serves  to  prevent  an  110V,  60  Hz  system  from  easily  interfacing  with  a  220V, 
50  Hz  supply. 

While  it  is  envisioned  that  much  of  the  experimental  development  of  intersystem  couplers 
will  be  to  first  order  independent  particular  SoS  architectures,  there  may  be  situations  that  call 
for  development  of  more  specific  couplers,  such  as  unique  lines  of  code  that  interface  legacy 
application  specific  software. 

There  are  three  key  products  of  intersystem  coupling  experimentation.  The  first  are 
protocols,  standards,  and  other  intersystem  couplers  that  form  the  tool  set  for  concept 
development  and  the  plug-in  hooks  for  rapid  SoS  field-testing.  As  these  mature  and  the  utility  of 
particular  intersystem  couplers  become  more  apparent,  a  second  set  of  products,  robust 
convergence  protocols,  evolves.  The  third  product  is  the  result  of  a  more  focused  consideration 
of  a  particular  SoS  concept:  a  set  of  requirements  on  each  component  system  for  incorporating 
specific  SoS  enabling  couplers.  Since  this  could  entail  substantial  downstream  modification  of  a 
particular  system’s  development  path,  the  component  systems  development  teams  must 
participate  in  the  development  of  this  set  of  products. 


Provisional  Fielding 

The  third  experimental  venue  for  SoS  exploration  is  provisional  fielding.  The  purpose  of 
this  venue  is  to  speed  transition  through  rapid  prototyping  of  SoS  enabling  intersystem  couplers 
in  a  set  of  component  systems.  These  experimentally  modified  versions  of  the  systems  can  then 
be  used  for  assessing  the  anticipated  SoS  benefit;  discovering  unexpected,  beneficial,  and 
detrimental  consequences  of  the  SoS  coupling;  and  providing  a  “first  spin”  look  at  the  impact  on 
the  constituent  systems  as  independent  elements.  Each  system  gets  a  “taste”  of  what  it  needs  to 
accommodate  to  achieve  SoS  interoperability,  a  definition  of  its  development  path  and  the 
benefit  it  derives. 

Provisional  fielding  can  incorporate  systems  developed  through  a  traditional  acquisition 
process  or  can  integrate  new  systems  in  early  development  that  have  been  generated  through 
non-traditional  approaches  such  as  Tactical  Exploitation  of  National  Capabilities  (TENCAP)  and 
Big  Safari.  The  intent  is  to  integrate  convergence  protocols  and/or  other  intersystem  couplers  as 
much  as  possible  as  “bolt-ons”  with  a  goal  of  minimal  invasiveness  on  the  systems. 
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Another  element  of  provisional  testing  is  engaging  all  stakeholders  (users,  developers,  the 
acquisition  community,  testers,  trainers,  sustainment/logistics  experts,  and  system  level 
researchers)  at  an  early  stage  in  the  eventual  SoS  development  and  deployment.  Collectively 
they  provide  input  that  guides  development  of  the  primary  products  of  provisional  testing.  A 
shortcoming  in  the  past  has  been  experimentation  and  early  deployment  that  has  not  adequately 
addressed  all  of  the  stakeholder  interests.  For  example,  without  adequate  training  coordination, 
initial  capability  can  often  disappear  when  the  first  users  rotate  out  to  other  assignments. 
Sustainment/Logistics  have  a  vital  role  in  ensuring  that  system  development  pays  heed  to 
component  “-ilities”:  availability,  serviceability,  upgradeability,  and  so  forth  (Global  Hawk 
example  of  obsolete  parts). 

Output  products  of  the  provisional  fielding  experimentation  are  user  validation,  initial 
operational  capability,  and  an  SoS  driven  acquisition  plan.  A  primary  focus  of  the  entire 
experimentation  process  is  to  generate  user  enthusiasm  that  translates  into  user  requirements  for 
the  SoS  capability.  Provisional  fielding  provides  confirmation  of  value  for  community  of 
interest  users  and  validates  concept  generated  users  pull/requirements. 

Provisional  testing  will  also  provide  quick  operational  capability.  Like  what  was 
envisioned  as  “left  behinds”  for  JEFX,  the  set  of  SoS  enabled  systems  (albeit  at  an  early  maturity 
stage)  provides  an  initial  operational  capability  and  test-bed  for  further  CONOPS 
experimentation.  A  third  product  is  a  “Go  Forward”  Development  Plan,  which  is  an  updated 
user-driven  requirements  document  that  can  feed  a  standard  SE  process  for  individual  system 
upgrades. 
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If  SoS  CONOPS  and  TTPs  p®| 
Were  A  vail  able. . .  %MJ* 


■  Joint  Unmanned  Combat  Air  Systems  (J-UCAS) 

■  Program  plans  were  to  conduct  a  number  of 
demonstrations  and  operational  assessments 

■  This  is  useful  for  future  anticipated  system  of 
system  constructs, 

■  ...  but  does  little  to  explore  future  capabilities 
and  liabilities  for  unanticipated  system  of 
system  constructs 

■  A  different  type  of  experimentation  venue 
needed  to  explore  other  system  of  system 
constructs? 

■  i.e.  develop  new  system  concepts  and 
associated  CONOPS,  TTPs  and  interfaces 
in  a  SoS  environment 

■  Challenging  scenarios,  non-designed  for 
missions,  willingness  to  fail 


An  example  of  a  system  development  that  could  benefit  from  experimentation  would  be 
the  J-UCAS  program.  The  J-UCAS  program  plans  to  conduct  several  operational  assessments 
(OAs)  during  the  2007-2010  timeframe.  One  of  the  goals  of  the  OA  is  to  demonstrate  that  DoD 
and  the  Air  Force  can  fully  integrate  J-UCAS  into  the  battlespace  in  a  system-of-systems 
environment  to  achieve  desired  effects,  not  merely  to  act  as  a  stand-alone  entity.  Two  key  areas 
for  these  assessments  are  communications/networking  and  vehicle  mission  control  systems.  This 
is  an  important  activity,  but  it  does  not  address  the  capabilities  and  liabilities  associated  with  the 
system’s  ability  to  operate  in  unanticipated  system-of-systems  constructs.  What  is  lacking  is  an 
experimentation  venue  focused  on  exploring  non-traditional  future  scenarios  where  the  Air  Force 
needs  other  system-of-systems  constructs  to  achieve  the  desired  effects.  Such  an 
experimentation  venue  would  allow  for  the  development  of  system  CONOPS,  TTPs,  and  the 
necessary  interfaces  required  to  insure  robust  future  system  capabilities. 

Developers  and  users  should  conduct  experimentation  in  a  challenging  environment,  not 
necessarily  focused  on  the  scenario  and  threats  used  for  development  of  the  initial  system 
requirements,  but  in  scenarios  where  they  can  tolerate  failure  as  a  means  whereby  they  can  learn 
lessons  and  better  define  future  requirements.  Combat  experience  shows  that  the  Air  Force  uses 
systems  and  interoperates  with  other  systems  in  ways  not  originally  anticipated.  Combat 
conditions  can  force  users  to  develop  interfaces  and  tactics  to  allow  systems  to  interoperate  in 
ways  not  originally  anticipated.  This  experimentation  would  serve  as  a  discovery  and  training 
venue,  reducing  the  need  to  accomplish  this  under  wartime  conditions,  thus  resulting  in  a  more 
capable  and  ready  force. 
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Barriers... 


■  Mythology 

■  “Traditional”  systems  engineering  processes 

■  The  bureaucracy  only  knows  one  way  to  do  things 

■  Need  all  resources  focused  on  proficiency  “Training” 

■  Reality 

■  FAR,  DoD  5000.2,  etc.  all  emphasize  satisfying  “User 
Requirements” 

■  Training  needs  to  be  balanced  with  Innovation 

■  innovation  is  really  a  form  of  training... 

■  we  always  use  innovation  in  combat;  people  need  training  in  how  to  innovate 
with  discipline 

■  Technology 

■  Need  “discovery”  mechanisms  to  achieve  full  Network  Centric 
Operations 

■  Need  means  of  quickly  identifying,  converging,  and 
promulgating  the  protocols  that  enable  system-of-systems 
interoperation 


For  any  innovation  there  are  usually  barriers  to  overcome.  In  the  case  of  the  proposed 
SoSE  methodology,  the  first  barrier  is  tradition.  The  incorporation  of  experimentation  venues 
for  refining  requirements  and  for  accelerating  product  fielding  runs  counter  to  the  existing 
culture.  It  is,  therefore,  important  for  the  bureaucracy  to  not  resist  the  idea  of  becoming  involved 
in  creating  and  facilitating  the  new  concept.  Having  experts  from  each  of  the  affected 
stakeholder  organizations  review  and  refine  the  methodology  can  help. 

The  second  barrier  is  limited  resources.  The  need  for  investment  in  training  continues.  It 
is  not  appropriate  to  send  inadequately  trained  people  into  combat  situations.  However,  one 
aspect  of  successful  combat  operation  is  the  ability  to  deal  with  innovation.  Hence,  the  new 
methodology  recommends  incorporation  of  “disruptive-innovation”  in  the  training  and 
experiment  process  for  just  that  reason:  To  prepare  warfighters  for  the  changing  conditions  of 
combat. 

Lastly,  there  are  some  technology  shortfalls.  The  most  notable  is  the  absence  of 
discovery  mechanisms  for  information  systems  networks.  Just  being  “on  the  network”  does  not 
achieve  the  vision  of  network  centricity.  Systems  participating  as  a  system-of-systems  need  to 
be  able  to  discover  the  existence  of  information  relevant  to  their  mission  without  having  that 
relationship  preprogrammed  several  years  prior  to  a  wartime  event.  Information  and  service 
discovery  needs  to  be  managed  “on  the  fly”  at  a  pace  that  coincides  with  the  “pick-up  game’ 
relationships  that  are  often  experienced  in  combat. 

The  other  technology  shortfall  is  the  refinement  of  convergent  protocols  necessary  to 
support  system-of-systems  interactions.  Currently  multiple  competing  middleware  solutions 
exist  from  various  agencies,  programs,  and  services  (e.g.,  the  Distributed  Common  Ground 
Systems  (DCGS)  Integration  Backbone,  Future  Combat  Systems  (FCS)  System-of-Systems 
Common  Operating  Environment,  Air  Operations  Center  (AOC)  web  services,  IBM  web-sphere 
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used  in  support  command  and  control  systems,  Hewlett-Packard’s  products,  and  so  forth). 
Successful  achievement  of  system-of-systems  enablement  will  require  convergence  of  these 
multiple  options  down  to  a  small  set  (or  even  only  one  set)  of  protocols  that  allow  information 
discovery,  access,  and  utilization  among  systems.  In  other  words,  merely  providing  “access”  via 
a  network  is  insufficient.  More  effort  is  required  for  the  protocols  to  converge. 
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4.  Research  Directions 

As  described  in  the  previous  sections,  System-of- Systems  Engineering  is  not  yet  a  well- 
defined  practice,  let  alone  a  formal  discipline.  Over  the  long-term,  it  is  clear  that  SoSE  must 
evolve  to  such  a  discipline  if  we  wish  to  effectively  conceive,  design,  implement,  and  operate 
interoperable  systems  that  enable  new  Air  Force  and  DoD  capabilities. 

We  can  compare  the  current  state  and  its  future  to  the  Systems  Engineering  discipline  in 
the  20th  century.  Initially,  complex  systems  were  built  through  trial  and  error,  with  many 
failures.  Eventually  exciting  new  products  and  capabilities  resulted.  As  the  field  matured, 
modeling  and  simulation  played  a  greater  role,  enabled  by  the  quantification  of  system 
performance  and  properties.  This  encouraged  more  rapid  (and  cheaper)  development  of  systems, 
at  higher  levels  of  complexity.  Finally,  a  discipline  of  Systems  Engineering  evolved.  This 
allowed  for  the  codification  of  the  process  required  to  determine  and  to  manage  system 
requirements.  This  in  turn  evolved  into  standard  training  courses  for  engineers.  Today,  we  are 
able  to  develop  systems  that  have  levels  of  performance,  robustness,  and  efficiency  previously 
thought  not  possible.  Military  aircraft  and  spacecraft  are  excellent  examples  of  the  achievements 
of  the  overall  Systems  Engineering  discipline. 

The  recommendations  of  this  report  reflect  our  belief  that  SoSE  is  not  yet  a  discipline, 
and  hence  experimentation  will  be  required  to  make  progress  in  the  short-term.  However,  over 
the  long-term,  additional  investments  in  SoSE  research  are  required  to  create  a  predictive 
engineering  discipline.  Academia,  government,  and  industry  will  carry  out  this  research.  In 
addition,  to  achieve  success,  this  research  must  likely  couple  these  players. 

A  first  step  is  research  aimed  at  quantifying  and  understanding  the  relationships  of  a  new 
set  of  “-ilities”.  These  new  “-ilities”  include  flexibility,  reconfigurability,  evolvability,  emerge- 
ability,  subscribe-ability,  and  others.  Metrics  and  analytic  tools  that  capture  the  “designer’s 
intent”  will  allow  better  consideration  of  these  properties  in  the  design  process,  as  well  as 
appropriate  trade-offs  that  are  central  to  good  engineering  design.  A  related  area  of  inquiry  seeks 
to  understand  how  to  exploit  the  DoD  value  chain  in  creation  of  SoSs  as  contrasted  with  the 
commercial  value  chain  that  is  driven  by  different  motivators  and  business  models. 

A  second  area  of  research  is  the  development  of  a  framework  and  tools  for  helping 
identify  convergence  protocols.  As  highlighted  in  the  last  section,  convergence  protocols  are  an 
enabler  for  rapid  innovation  and  parallel  development  of  interoperable  technologies  that  lead  to 
unanticipated  synergies.  New  approaches  are  required  to  understand  how  to  model  systems-of- 
systems  at  an  appropriate  level  of  abstraction  in  which  such  convergence  protocols  can  be 
identified  and  evaluated.  If  such  tools  can  be  developed,  they  will  allow  us  to  move  from  a 
reliance  on  experimentation  (or  market-forces  for  commercial  applications)  to  a  more  analytical 
basis  for  understanding  tradeoffs  and  selecting  standards  that  will  become  convergence 
protocols. 

Finally,  an  important  element  of  System-of- Systems  Engineering  for  DoD  systems  is  the 
human  element.  Research  in  human  system  interaction  (HSI)  and  decision-making  is  required  to 
understand  better  how  to  integrate  these  elements  into  an  effective  system-of-systems 
architecture. 
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The  traditional  research  environment  does  not  effectively  support  large  scale, 
transdisciplinary  research;  yet  such  research  is  critical  to  developing  the  discipline  of  SoSE.  The 
traditional  research  environment  emphasizes  disciplines,  but  the  issues  that  the  Air  Force  faces 
do  not  break  along  clean  disciplinary  lines.  Rather  SoSE  research  demands  a  high  degree  of 
collaboration  between  government,  industry,  and  academia.  It  requires  new  approaches  to  how 
the  Air  Force  allocates  research  funds,  how  research  partners  collaborate,  and  how  the  Air  Force 
can  fully  realize  research  synergies. 

Research  in  SoSE  involves  experimentation,  modeling  and  simulation,  as  well  as  field 
studies  that  contribute  to  new  knowledge  on  the  optimal  structures  and  behaviors  of  complex 
systems-of-systems.  The  Air  Force  can  transition  this  research  to  practice  in  the  form  of  new 
methodologies  and  enabling  technologies,  as  well  as  the  development  of  case  studies  for  use  in 
educating  the  workforce.  The  desired  impact  of  the  research  is  knowledge  and  enabling 
practices  and  technologies  for  developing  SoSs  with  outcomes  that  are  more  predictable. 

The  research  agenda  is  still  emerging  for  SoSE;  however,  research  is  already  underway  at 
some  leading  universities,  working  in  partnership  with  government  and  industry.  Several 
universities  have  established  crosscutting  programs,  research  partnerships,  and  new  types  of 
laboratories  to  tackle  the  challenges  of  systems-of-systems. 

While  not  the  focus  of  this  report,  the  reader  should  note  that  the  effective 
implementation  of  the  SoSE  process  depends  upon  the  enabling  environment  to  support  it.  This 
includes  acquisition  practices  that  are  suited  to  the  nature  of  the  SoS,  technologies  and  toolsets 
that  enable  engineers  to  perform  SoSE,  and  solving  some  of  the  pragmatic  business  concerns 
related  to  SoS  endeavors. 

Many  enablers  already  exist.  However,  we  need  more  before  we  can  realize  the  SoSE 
vision.  Enablers  include  the  many  policies  and  guidance  documents  promoting  the  importance 
of  SE  and  SoSE  and  various  standards  and  guidance  documents  that  have  improved  architecting 
and  engineering  practices.  There  is  also  a  shared  mindset  that  good  Systems  Engineering 
processes  need  to  be  applied  to  the  component  systems  in  the  SoS. 

Many  barriers  exist,  the  first  of  which  is  that  there  is  not  yet  a  prevailing  mindset  that 
SoSE  is  more  than  traditional  SE  applied  to  a  larger  system;  we  must  overcome  this  as  a  first  step 
to  a  robust  SoSE  discipline.  Many  of  the  present  acquisition  practices,  while  very  effective  for 
traditional  SE,  demand  levels  of  detail  or  levels  of  technology  readiness  too  early  in  the  life  cycle 
for  an  SoS.  New  practices  are  needed  that  will  ensure  the  necessary  level  of  control,  but  will 
better  accommodate  the  (necessary)  uncertainty  involved  in  SoSs.  Another  barrier  is  the  lack  of 
protocols  and  a  modeling  environment  for  pulling  together  independently  generated  system 
models  so  that  these  can  interact  together  in  larger  scale  experimentation.  Some  very  significant 
barriers  exist  that  are  rooted  in  the  realities  of  the  competitive  marketplace.  We  lack  the 
strategies  for  dealing  with  IP  issues  and  competitive  advantage  concerns  in  the  complex  SoS 
endeavors  where  many  organizations  are  collaborating  in  a  way  where  open  technical  exchange 
is  essential  to  success.  We  need  additional  studies  to  understand  SoS  enablers  and  barriers  more 
fully. 
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5.  Summary 


SE  Process 


Shown  here  is  the  current  system  development  process.  This  process  starts  with  a  clear 
understanding  of  the  military  user  needs  from  which  system  requirements  are  derived.  It  is  well 
recognized  that  system  requirements  play  a  preeminent  role  in  the  Systems  Engineering  process 
and  subsequent  acquisition  processes  that  lead  to  a  fielded  capable  system. 
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SE  Process 


The  acquisition  of  any  weapon  system  is  a  complicated  process.  The  Joint  Capabilities 
and  Integration  Development  System  (JCIDS)  focuses  on  identifying  user  needs  and  conducting 
the  necessary  analysis  to  determine  the  best  material  or  non-material  means  to  overcome  an 
identified  capabilities  gap.  The  JCIDS  process  also  focuses  on  meeting  joint  warfighter 
requirements  and  closely  integrates  with  major  milestone  decisions. 
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We  propose  experimentation  venues  as  a  means  for  achieving  pragmatic  near-term 
results.  As  previously  described,  these  venues  should  support  disruptive  and  innovative 
CONOPS  experiments,  convergence  protocol  evaluation  and  selection,  and  support  for  rapid 
provisional  fielding  of  SoS  enabled  systems.  Also  from  a  pragmatic  approach,  technology  that 
provides  discovery  among  systems  and  makes  association  of  information  needs  among  the 
publishers  and  subscribers  is  necessary  to  achieve  the  full  potential  of  network  centric 
operations. 

The  addition  of  an  experimentation  venue  to  the  process  will  allow  for  the  development 
and  articulation  of  user  needs  specially  focused  on  achieving  system-of-systems  capabilities. 
The  lessons  learned  from  the  experimentation  venue  will  drive  user  needs  as  well  as  specific 
system  requirements,  which  can  assist  the  JCIDS  process  by  providing  experimentation  derived 
user  requirements  to  support  the  development  of  the  Initial  Capabilities  Document  (ICD).  As 
shown  here,  the  suggested  SoSE  processes  are  compatible  with,  but  not  dependent  upon,  the 
JCIDS  process.  Thus,  while  JCIDS  can  benefit  from  SoSE,  the  SoSE  processes  will  continue  as 
a  meaningful  and  vital  part  of  systems  development  even  if  DoD  replaces  JCIDS  with  something 
else. 
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SoSE  Process 


To  encourage  the  use  of  SoSE,  the  government  should  undertake  specific  SoSE  research 
initiatives  to  promote  investigation  within  this  topic.  There  are  aspects  of  such  investigation  that 
would  be  applicable  to  both  commercial  and  DoD  environments,  but  there  are  some  aspects  of 
such  investigation  that  must  be  focused  on  DoD  (and  USAF)  unique  constraints.  The  differences 
in  the  enterprise  environment  between  commercial  interests  and  DoD  legal  constants  can  have  a 
profound  effect  upon  the  direction  of  research.  Only  DoD  sponsored  research  is  likely  to 
produce  results  that  are  usable  within  DoD. 

The  results  from  these  research  initiatives  will  directly  feed  into  the  experimentation 
venues  such  that  the  results  of  the  initiatives  can  be  tested  and  evaluated  in  a  system-of-systems 
environment  complete  with  user  involvement  allowing  for  the  development  of  new  enabling 
CONOPS  and  TTPs. 

Research  initiatives  that  are  focused  on  further  developing  System-of-Systems 
Engineering  as  a  practice,  and  ultimately  as  a  discipline,  will  further  enhance  the  system 
development  process,  by  developing  a  true  System-of-Systems  Engineering  process. 
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The  fundamental  change  in  JCIDS  from  the  previous  requirements  system  is  to  base  it  on 
a  top-down  analyses  rather  than  a  bottom-up  requirements  generation. 

The  JCIDS  analyses  result  from  the  national  security  strategy  and  overarching  concepts. 
From  these  overarching  concepts,  joint  operating  concepts  and  functional  concepts  result  along 
with  integrated  architectures  that  pull  together  these  concepts  and  the  associated  systems. 

The  intent  of  this  top-down  approach  is  to  identify  capability  gaps,  ensure  development 
of  new  capabilities  within  a  joint  warfighting  context,  and  increase  the  number  of  systems  that 
are  actually  “bom  joint”  in  the  first  place. 
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6.  Recommendations 


Recommendations 


■  Enable  experimentation 

■  Experiment  for  CONOPS  development...  to  build  innovative  TTPs, 
and  to  build  trust  among  cross  domain  participants  (Action: 
AF/XO?,  ACC?) 

■  Experiment  for  infrastructure  advances...  sort  out  the 
implementable  technology  for  the  middleware  layer  that  enables  Dl 
(Action:  SAF/AQ?  AFRL?  SAF/XC?) 

■  Experiment  venue  as  a  transition  venue....  Allow  “provisional” 
fielding  (Action:  SAF/AQ?) 

■  Pursue  technologies  for  SoS  (Action:  AFRL  ICC) 

■  Support  research  into  theory  and  discipline  of  SoSE  (Action:  AFOSR) 

■  Form  an  IPT  with  membership  from  each  of  the  USAF  stakeholders 
(ACC,  AFSPC,  SAF/AQ,  AFOTEC,  AETC,  AFMC,  AFRL)  to  influence 
internal  management  practices  that  foster  the  new  SoSE  Methodology 
(Action:  SAF/XC) 


A  Pragmatic  Approach  to  SoSE... 


There  are  four  recommendations  from  the  study. 

First,  set  up  three  experimentation  venues  under  the  auspices  of  three  different 
organizations.  It  is  important  that  there  be  separate  organizations  owning  the  experimentation 
venues  so  that  each  venue  preserves  its  identity.  The  purposes  for  which  the  venues  serve  need 
separate  sponsorship  and  advocacy.  On  the  other  hand,  it  is  also  important  for  the  three  venues 
to  work  together.  The  interplay  among  each  experimentation  venue  enables  the  effective 
operation  of  the  other  two.  We  recommend  Air  Combat  command  and  the  Deputy  Chief  of  Staff 
of  the  Air  Force  for  Plans  and  Operations  (AF/XO)  pursue  the  CONOPS  experimentation  venue. 
The  Air  Force  Research  Lab,  the  Assistant  Secretary  of  the  Air  Force  for  Acquisition  (SAF/AQ), 
and  Secretary  of  the  Air  Force,  Office  of  Warfighting  Integration  and  Chief  Information  Officer 
(SAF/XC)  should  work  together  to  pursue  infrastructure  experimentation.  SAF/AQ  should  also 
pursue  experimentation  for  provisional  fielding. 

Second,  pursue  the  two  technologies  described  in  the  report.  Discovery  agents  are 
necessary  to  enable  network  centricity  versus  merely  network  enablement.  Moreover, 
disciplined  evaluation  of  convergence  protocols  is  necessary  to  identify  the  expected  standards 
for  implementation  in  every  system  that  is  to  be  “system-of-systems  enabled.”  We  recommend 
responsibility  for  this  go  to  the  Commander,  Air  Force  Research  Laboratory  (AFRL/CC). 

Third,  invest  further  in  the  underlying  theory  of  interactions  among  systems  to 
accomplish  the  predictable  analysis  of  systems  engineered  as  part  of  a  system-of-systems.  This 
reflects  the  strong  pragmatic  perspective  of  this  report  and  the  resultant  methodology.  We 
believe  responsibility  for  this  should  go  to  the  Air  Force  Office  of  Scientific  Research  (AFOSR). 
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Fourth,  assemble  an  integrated  product  team  (IPT)  of  the  affected  bureaucracies  to  refine 
this  recommended  methodology  so  that  the  bureaucracies  can  incorporate  the  methodology 
within  their  respective  organizations.  We  recommend  that  SAF/XC  should  take  responsibility 
for  forming  the  IPT.  Further,  we  recommend  that  IPT  membership  should  include  the  following: 

Air  Combat  Command  (ACC) 

Air  Force  Space  Command  (AFSPC) 

Assistant  Secretary  of  the  Air  Force  for  Acquisition  (SAF/AQ) 

Air  Force  Operational  Test  and  Evaluation  Center  (AFOTEC) 

Air  Education  and  Training  Command  (AETC) 

Air  Force  Materiel  Command  (AFMC) 

Air  Force  Research  Laboratory  (AFRL) 
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Appendix  A:  Terms  of  Reference 

USAF  SCIENTIFIC  ADVISORY  BOARD  2005  QUICK  LOOK  STUDY 
System-of-Systems  Engineering  for  Air  Force  Capability  Development 

Terms  of  Reference 


BACKGROUND 

Inadequate  application  of  Systems  Engineering  principles  and  processes  is  the  rationale  used  to 
explain  why  acquisitions  are  behind  schedule,  over  budget,  and  deficient  in  required 
functionality.  Yet,  the  current  state  of  systems  engineering  does  not  adequately  support  the 
development  of  complex,  adaptive,  and  software-intensive  system-of-systems  (SoS)4  in  which 
humans  are  parts  of  the  system.  While  capabilities-based  justification  of  operational  need,  such 
as  supported  in  the  CRRA  process,  is  a  step  in  the  right  direction,  there  is  no  well-established 
SoS  methodology  and  associated  tools  and  techniques  that  can  support  faster  engineering 
analysis  and  realization  of  required  capabilities.  We  need  a  methodology  that  can  match 
operational  tempo  -  that  can  quickly  field  ‘good  enough’  systems  that  can  be  further  developed 
and  supported  concurrent  with  their  operational  test  and  use.  The  existing  tools  and  processes 
are  often  focused  on  a  very  limited  number  of  narrow,  pre-defmed  alternatives  and  lack  the 
fidelity,  agility,  and  integration  necessary  to  provide  responsive,  comprehensive  analysis  of 
alternatives.  The  Air  Force  needs  to  build  an  understanding  of  the  critical  developmental  and 
research  needs  in  the  (system)  engineering  of  systems-of-systems. 

STUDY  PRODUCTS 

Briefing  to  SAF/OS  &  AF/CC  by  August  2005.  Publish  report  by  December  2005. 

CHARTER 

This  quick  look  study  will  propose  an  engineering  methodology,  tailored  for  use  by  the  Air 
Force,  for  software  intensive  SoS  development  with  the  dual  goals  of  engineering  a  robust  and 
adaptable  SoS  that: 

(1)  Provides  validated  operational  capabilities;  and 

(2)  Is  delivered  at  a  cycle  time  synchronous  with  current  operational  tempo. 

The  engineering  methodology  will  leverage: 

(1)  Existing  (and  sometimes  unused)  sound  systems  engineering  principles, 


4  A  system  will  be  called  a  System-of-Systems  (SoS)  when: 

The  component  systems  achieve  well-substantiated  purposes  in  their  own  right  even  if  detached 
from  the  overall  system; 

The  components  systems  are  managed  in  large  part  for  their  own  purposes  rather  than  the 
purposes  of  the  whole; 

It  exhibits  behaviors  (including  emergent  ones)  not  achievable  by  the  component  systems  acting 
independently; 

Functions,  behaviors  and  component  systems  may  be  added  or  removed  during  its  use. 
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Appendix  A:  Terms  of  Reference  (continued) 

(2)  Existing  (and  sometimes  very  successful)  creative,  non-traditional,  innovation  driven 
acquisition  processes, 

(3)  Operator  derived  expectations  for  systems  engineering  outcomes  as  exhibited  among 
warfighters  with  field  experience  on  SoS  solutions,  and 

(4)  Evolving  research  results  in  executable,  model-based  architecture  to  support 
concurrent  discovery  of  requirements,  simulatable  and  testable  SoS  representations, 
analysis  and  design  of  SoS  architecture,  and  rapid  transformation  into  fieldable 
capabilities. 

The  study  will  be  scoped  to  focus  on  an  exemplar  area  based  on  Air  Force  needs:  e.g.,  (1)  the 
integration  of  information  operations  capabilities  into  the  CAOC;  or  (2)  use  of  modeling  and 
simulation  for  joint  operational  testing. 
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Appendix  B:  Study  Members 

Study  Leadership 

Mr.  Thomas  “Skip”  Saunders 

Study  Panel 

Dr.  Wanda  Austin 
Dr.  John  Brock 
Mrs.  Natalie  Crawford 
Dr.  Mica  Endsley 
Mr.  Ed  Glasgow 
Dr.  Dan  Hastings 
Prof.  Alex  Levis 
Prof.  Richard  Murray 
Dr.  Donna  Rhodes 
Dr.  Marvin  Sambur 
Ms.  Heidi  Shyu 
Mr.  Phil  Soucy 
Dr.  David  Whelan 

General  Officer  Participant 

Lt  Gen  (S)  Charles  Croom,  USAF 

Study  Management 

Maj  Christopher  Berg,  USAF  -  Project  Manager 
Maj  Alan  Seraile,  USAF  -  Executive  Officer 
Maj  David  Herring,  USAF  -  Technical  Writer 
Mr.  Justin  Waters  -  Analyst 
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ACC 

AETC 

AF 

AF/XO 

AFC2ISRC 

AFMC 

AFOSR 

AFOTEC 

AFRF 

AFRF/CC 

AFSPC 

AOC 

Arc,  Arch 

AT&L 

AT&T 

BFUF 

C2 

C4ISR 

CAIG 

CDD 

CIO 

CJCSI 

COCOM 

CONOPS 

CONPFANS 

CPD 

CWIN 

DAB 

DARPA 


Appendix  C:  Acronyms  and  Abbreviations 

Air  Combat  Command 

Air  Education  and  Training  Command 

Air  Force 

Deputy  Chief  of  Staff  of  the  Air  Force  for  Plans  and  Operations 

Air  Force  Command,  Control,  Intelligence,  Surveillance,  and 
Reconnaissance  Center 

Air  Force  Materiel  Command 

Air  Force  Office  of  Scientific  Research 

Air  Force  Operational  Test  and  Evaluation  Center 

Air  Force  Research  Faboratory 

Commander,  Air  Force  Research  Laboratory 

Air  Force  Space  Command 

Air  Operations  Center 

Architecture 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
American  Telephone  and  Telegraph 
Bottom  Line  Up  Front 
Command  and  Control 

Command,  Control,  Communications,  Computers,  Intelligence, 
Surveillance,  and  Reconnaissance 

Cost  Analysis  Improvement  Group 

Capability  Development  Document 

Chief  Information  Officer 

Chairman  Joint  Chief  of  Staff  Instruction 

Combatant  Command,  Combatant  Commander 

Concept  of  Operations 

Contingency  Plans 

Capability  Production  Document 

Cyber  Warfare  Integration  Network 

Defense  Acquisition  Board 

Defense  Advanced  Research  Projects  Agency 
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Appendix  C:  Acronyms  and  Abbreviations  (continued) 


DAS 

Defense  Acquisition  System 

DCGS 

Distributed  Common  Ground  Systems 

DII-COE 

Defense  Integration  Infrastructure  -  Common  Operating  Environment 

DISA 

Defense  Information  Systems  Agency 

DMOC 

Distributed  Mission  Operations  Center 

DoD 

Department  of  Defense 

DoDD 

Department  of  Defense  Directive 

DoDI 

Department  of  Defense  Instruction 

DOTMLPF 

Doctrine,  Organization,  Training,  Materiel,  Leadership  &  education, 
Personnel  &  Facilities 

DSP 

Defense  Support  Program  Satellite 

DVD 

Digital  Video  Disc 

E2E 

End-to-End,  Enterprise-to-Enterprise 

ES 

Enterprise  Services 

ESC 

Electronic  Systems  Center 

ESC-CC 

Electronic  Systems  Center-Commander 

ESC-EN 

Electronic  Systems  Center-Engineering 

FAR 

Federal  Acquisition  Regulations 

FCS 

Future  Combat  Systems 

FFRDC 

Federally  Funded  Research  and  Development  Center 

FTP 

File  Transfer  Protocol 

GIG 

Global  Information  Grid 

HD 

High  Definition 

HDLC 

High-level  Data  Link  Control 

HLA 

High  Level  Architecture 

HSI 

Human  Systems  Interaction 

HTTP 

Hyper  Text  Transfer  Protocol 

Hz 

Hertz 

IA 

Information  Assurance 

ICD 

Initial  Capabilities  Document 

IEEE 

Institute  of  Electrical  &  Electronic  Engineers 

PUBLIC  RELEASE 
80 


PUBLIC  RELEASE 


Appendix  C:  Acronyms  and  Abbreviations  (continued) 


IETF 

Internet  Engineering  Task  Force 

Imp 

Implementation 

IOC 

Initial  Operational  Capability 

IP 

Internet  Protocol 

IPL 

Integrated  Priority  List 

IPT 

Integrated  Product  Team 

IPv6 

Internet  Protocol  version  6 

IT 

Information  Technology 

ITAB 

Information  Technology  Acquisition  Board 

ITU 

International  Telecommunication  Union 

JBI 

Joint  Battlespace  Infosphere 

JCIDS 

Joint  Capability  Integration  &  Development  System 

JEFX 

Joint  Expeditionary  Force  Experiment 

JPEG 

Joint  Photographic  Experts  Group 

JROC 

Joint  Requirements  Oversight  Council 

JROCM 

Joint  Requirements  Oversight  Council  Memorandum 

J-UCAS 

Joint  Unmanned  Combat  Air  Systems 

KIPs 

Key  Interface  Profiles 

KPP 

Key  Performance  Parameter 

KPPs 

Key  Performance  Parameters 

MA 

Mission  Area 

MS-A 

Milestone  A 

MS-B 

Milestone  B 

MS-C 

Milestone  C 

NC 

Net  Centric,  Netcentric 

NCID 

Net-Centric  Implementation  Document 

NCOIC 

Network  Centric  Operations  Industry  Consortium 

NCOW-RM 

Net-Centric  Operations  and  Warfare 

NCW 

Network  Centric  Warfare,  Net  Centric  Warfare 

NESI 

Netcentric  Enterprise  Solutions  for  Interoperability 

NetBEUI 

NetBIOS  Extended  User  Interface 
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Appendix  C:  Acronyms  and  Abbreviations  (continued) 


NetOps 

Network  Operations 

Nil 

Assistant  Secretary  of  Defense  for  Networks  and  Information  Integration 

NR-KPP 

Net-Ready  Key  Performance  Parameter 

NSS 

National  Security  System,  National  Security  Strategy 

OAs 

Operational  Assessments 

OEF 

Operation  Enduring  Freedom 

OPLANS 

Operations  Plans 

Ops 

Operations 

OSD 

Office  of  the  Secretary  of  Defense 

PA&E 

DoD  Office  of  Program  Analysis  and  Evaluation 

PKI 

Public  Key  Infrastructure 

R&D 

Research  and  Development 

RA 

Resource  Analysis,  Risk  Assessment 

RAAR 

Responsibility,  Authority,  Accountability,  and  Resources 

RJ11 

Registered  Jack  1 1 

SA 

Situational  Awareness 

SAB 

Scientific  Advisory  Board 

SAE 

Society  of  Automotive  Engineers 

SAF/AQ 

Assistant  Secretary  of  the  Air  Force  for  Acquisition 

SAF/AQRE 

Assistant  Secretary  of  the  Air  Force  for  Acquisition,  Deputy  Assistant 
Secretary  (Engineering  and  Technical  Management  Division) 

SAF/XC 

Secretary  of  the  Air  Force,  Office  of  Warfighting  Integration  and  Chief 
Information  Officer 

SE 

Systems  Engineering 

SMTP 

Simple  Mail  Transfer  Protocol 

SoS 

Sy  stem-of-  Systems 

SoSE 

System-of-Systems  Engineering 

SVGA 

Super  Video  Graphics  Array 

TCP 

Transmission  Control  Protocol 

TCP/IP 

Transmission  Control  Protocol/Intemet  Protocol 

TENCAP 

Tactical  Exploitation  of  National  Capabilities 

TTPs 

Tactics,  Techniques  and  Procedures 
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Appendix  C:  Acronyms  and  Abbreviations  (continued) 

UDP  User  Datagram  Protocol 

USAF  United  States  Air  Force 

USB  Universal  Serial  Bus 

V  Volt 

VHS  Vertical  Helical  Scan  (or  as  JCV  calls  it,  "Video  Home  System") 
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Appendix  D:  Visits  and  Briefings 


Air  Force 

Deputy  Assistant  Secretary  of  the  Air  Force  for  Science,  Technology,  and  Engineering 
Air  Combat  Command 
Air  Force  Space  Command 

Air  Force  Command,  Control,  Intelligence,  Surveillance,  and  Reconnaissance  Center 
Air  Force  Electronic  Systems  Center 

Department  of  Defense 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
Assistant  Secretary  of  Defense  for  Networks  and  Information  Integration 
Office  of  Program  Analysis  and  Evaluation,  Cost  Analysis  Improvement  Group 
Missile  Defense  Agency 

Department  of  Homeland  Security 

U.S.  Coast  Guard 

Federally  Funded  and  Non-Profit  Organizations 

The  Aerospace  Corporation 
The  MITRE  Corporation 

System  of  Systems  Engineering  Center  for  Excellence  (SoSE  Conference) 

Industry 

The  Boeing  Company 
Lockheed  Martin  Corporation 
Northrop  Grumman  Corporation 

Universities 

California  Institute  of  Technology 
George  Mason  University 
Massachusetts  Institute  of  Technology 
Old  Dominion  University 
Purdue  University 
Stevens  Institute  of  Technology 
University  of  California  San  Diego 
University  of  Southern  California 
University  of  Virginia 
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Appendix  E:  Initial  Distribution 


Air  Force  Leadership 

Secretary  of  the  Air  Force 
Chief  of  Staff  of  the  Air  Force 
Under  Secretary  of  the  Air  Force 
Vice  Chief  of  Staff  of  the  Air  Force 

Air  Force  Secretariat 

Assistant  Secretary  of  the  Air  Force  for  Acquisition 

•  Deputy  Assistant  Secretary  of  the  Air  Force  for  Science,  Technology,  and  Engineering 
Secretary  of  the  Air  Force,  Office  of  Warfighting  Integration  and  Chief  Information  Officer 

•  Air  Force  Command,  Control,  Intelligence,  Surveillance,  and  Reconnaissance  Center 

Air  Staff 

Assistant  Vice  Chief  of  Staff  of  the  Air  Force 

Deputy  Chief  of  Staff  of  the  Air  Force  for  Air  and  Space  Operations 

Deputy  Chief  of  Staff  of  the  Air  Force  for  Plans  and  Operations 

Deputy  Chief  of  Staff  of  the  Air  Force  for  Plans  and  Programs 

Deputy  Chief  of  Staff  of  the  Air  Force  for  Test  and  Evaluation 

Director  of  the  Air  National  Guard 

Chief  of  Air  Force  Reserve 

Chief  Scientist  of  the  Air  Force 

Scientific  Advisory  Board  Military  Director 

Air  Force  Major  Commands 

Air  Combat  Command 

•  ACC  Chief  Scientist 

Air  Education  &  Training  Command 
Air  Force  Materiel  Command 
Air  Force  Reserve  Command 
Air  Force  Space  Command 

•  Space  Warfare  Center 
Air  Force  Special  Ops  Command 
Air  Mobility  Command 
Pacific  Air  Forces 

U.S.  Air  Forces  in  Europe 
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Appendix  E:  Distribution  (continued) 

Other  Air  Force  Elements 

Aeronautical  Systems  Center 
Air  Force  Electronic  Systems  Center 
Air  Force  Institute  of  Technology 

•  Center  for  Systems  Engineering 
Air  Force  Office  of  Scientific  Research 
Air  Force  Operational  Test  and  Evaluation  Center 
Air  Force  Research  Laboratory 
Air  Warfare  Center 
Space  and  Missile  Systems  Center 

Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
Assistant  Secretary  of  Defense  for  Networks  and  Information  Integration 
Office  of  Program  Analysis  and  Evaluation,  Cost  Analysis  Improvement  Group 

Joint  Chiefs  of  Staff 

Joint  Chiefs  of  Staff,  Director  of  C4  Systems 

Joint  Chiefs  of  Staff,  Director  of  Operational  Plans  and  Interoperability 
Defense  Agencies 

Defense  Information  Systems  Agency 
Missile  Defense  Agency 

Advisory  Boards 

Army  Science  Board 

Defense  Policy  Board 

Defense  Science  Board 

Naval  Research  and  Advisory  Committee 

Naval  Studies  Board 

Department  of  Homeland  Security 

U.S.  Coast  Guard 

Federally  Funded  and  Non-Profit  Organizations 

The  Aerospace  Corporation 
The  MITRE  Corporation 

System  of  Systems  Engineering  Center  for  Excellence 
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Appendix  E:  Distribution  (continued) 

Industry 

The  Boeing  Company 
Lockheed  Martin  Corporation 
Northrop  Grumman  Corporation 

Universities 

California  Institute  of  Technology 
George  Mason  University 
Massachusetts  Institute  of  Technology 
Old  Dominion  University 
Purdue  University 
Stevens  Institute  of  Technology 
University  of  California  San  Diego 
University  of  Southern  California 
University  of  Virginia 
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military  leaders  called  upon  for  action.  The  reality  is  that  the  Air  Force  does  not  build  all  systems  through  a  homogenous 
acquisition  and  development  process,  it  does  not  use  all  systems  in  ways  foretold  at  their  inception,  and  not  all  systems  find 
themselves  used  among  predicted  interface  partners.  Especially  in  wartime,  the  exigencies  of  war  sometimes  force  a 
reconfiguration  among  systems  or  even  demand  systems  behave  in  ways  that  create  new  capabilities.  When  such  changes  occur,  the 
users  in  the  field  oftentimes  find  the  tasks  associated  with  reengineering  interconnections  among  systems  falls  upon  them. 
Increasingly,  awareness  of  the  need  to  support  fungible  interconnection  among  systems  has  driven  the  Air  Force  and  systems 
engineers  to  start  thinking  about  the  demands  of  system-of-systems  configurations  and  the  engineering  issues  associated  with 
building  and  supporting  them. 

The  “System-of-Systems  Engineering  for  Air  Force  Capability  Development”  study  was  chartered  to  address  the  challenge 
of  developing  systems-of-systems  that  are  more  effective.  The  study  panel  conducted  this  study  in  response  to  a  request  by  the 
Secretary  of  the  Air  Force  and  the  Chief  of  Staff  of  the  Air  Force. 
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